ProxySG & Advanced Secure Gateway

Hi Team,

Our customer is facing high CPU, while we analysis the issue with the help of CPU monitor, we observed that the SSL and cryptography protocol consumes nearly 92% of CPU.

We have collected only sysinfo and evenlog in the time of high CPU. Once we rebooted the device it back to normal CPU was around 10%.

Is there any logs needs to collect in order to investigate this issue (unfortunatly we have rebooted the device).  Can we collect logs which is related to this cpu issue even after rebooted?

Please advice us to check and fix this issue.

SGOS: 6.6.4.2

Model: S200-20

Thanks,

Ram.

Hi Ram,

            Check the article https://support.symantec.com/en_US/article.TECH245157.html which is having some guidelines to deal with high CPU on SSL. If CPU remains even after performing these changes, do collect below and raise a TAC case.

Sysinfo

Eventlog

Snsphot_sysinfo

Snsphot_sysinfo_stats

Full Core (To be collected when CPU is very high) - Ref: https://support.symantec.com/en_US/article.TECH244735.html

Hi Aravind,

Thank you for the information.

But after rebooting the proxy CPU back to normal. We have taken only sysinfo and eventlogs at the time of high CPU.

Any other logs can collect which can help after rebooting. (customer already rebooted the device and requeted for RCA for the high CPU)

Thanks,

Ram.

Hi Ram,

             After the customer performing his troubleshooting (i.e. reboot), it is very difficult to provide an RCA. The files that you have might not say the exact issue. So analysis in such cases will be best effort. Inform customer to involve you on next occurrence. Do make sure that you add all the optimization provided in the article. Also an upgrade is due when SGOS version is concerned.

Hi Team,

The high CPU has resolved after followed the step in the below KB. and Disable SSL cache objects refreshing. After followed those steps CPU became less than 20%

https://support.symantec.com/en_US/article.TECH245157.html 

Disable SSL cache objects refreshing

# conf t
#(config) http no cache expired
#(config) http no pipeline client requests
#(config) http no pipeline client redirects
#(config) http no pipeline prefetch requests
#(config) http no pipeline prefetch redirects
#(config) http strict-expiration refresh
#(config) http strict-expiration serve

Thanks,

Ram.

Our customer is also experiencing high CPU utilization on SSL and Cryptography. TAC looked at it and recommended to increase emulated certificate cache timeout to 72 hours and disable SSL intercept on Google sites and Web Ads/Analytics category. Since customer is doing Transparent Authentication, we still need SSL intercept. However, since unauthenticated users accessing HTTPS are getting failed to authenticate tunneled SSL request, TAC recommended to enable Detect Protocol on External HTTP Proxy Service. Is this OK? I usually enable Detect Protocol when running Explicit so I'm not really sure with Transparent. Also, is it OK to lower emulated certificate key size as indicated in https://support.symantec.com/en_US/article.TECH245157.html?

Hi Mark,

             Even I am not sure on how enabling DP on External HTTP (i.e. TCP port 80) is related to this issue. Please check with TAC on the reason for this suggestion.

             For SSL high CPU issues, we touch the below first. These are mentioned in the article that you pointed

1. Increase SSL Cert cache timeout
2. Reduce Emulated Cert size to 1024
3. Enable splash text in SSL Intercept policy if you are still in an old version (6.5.9.9 or older in 6.5.x branch)
4. Disabling SSL Interception for some highly used sites

While the first 3 from the list is easy to achieve, 4th will need extra steps to make sure that you are not breaking the communication. SSL interception is a must for the proxy to perform the redirect for authentication. Hence bypassing SSL with restrict the ability to authenticate them. One workable solution would be to use surrogate based authentication such as "Origin-IP-Redirect" in which the chances of these bypassed domains challenged for authentication is very low.

TAC emailed back saying it won't do anything so I disabled DP on External HTTP. I'll try to do #2 since #1 was changed yesterday, #3 is not applicable (ProxySG is running 6.7.3.5) and in #4 we are intercepting 5 URL Categories to increase chances of getting authenticated. Can't disable intercept on Search  Engines/Portals and Technology/Internet as we noticed that Chrome and IE make background requests when launched and those requests falls on those two categories.

Hi Mark,

               Intercepting 5 categories will not force us to have authenticated for each new session provided we are on "Origin-IP-Redirect". So the chances of the backend/CDN url requests might escape from an authentication redirect. You may want to test this for some client IPs to see how it benefits before making it enabled for all.