Video Screencast Help

SEP 12.1.x causing slow log-off with roaming profiles

Created: 27 Nov 2013 • Updated: 27 Nov 2013 | 25 comments

Hi All,

We have been having an issue with SEP cauing slow log off times with our Windows 7 computers.

We believe that SEP is locking certain files in the users roaming profile during the log-off process. So when the files go to synchronise back to the file server where the roaming profiles are stored it is getting an access denied error (we found this via wireshark). It looks like that during the log off process it continually tries to access the locked files then after a few minutes it will log off (all files seem to synchronise successfully so not sure if it's timing out or just finally ignores SEP and takes ownership).

If we uninstall the SEP client the the computer logs off almost instantly.

We have created a test group in the SEPM and disabled all features and policies to see if we could find what was locking the files, but even after turning off all features including auto-protect the issue still occured.

Also adding the users profile folder as an exception did not make a difference (both as a policy and locally on the client).

Have even tried updating to the latest client 12.1.4013.4013

We're not too sure how long this has been going on for, but from user reports about a month. We have not changed anthing within our manager to correlate with this.

 

I have attached 2 events that occur during the prolonged log-out. Although we are not sure if they are related to SEP.

 

Any help would be much appreciated.

SEPM Version: 12.1.2015.2015 (this version running for about a year)

SEPM running on Windows Server 2008 R2

Client versions: 12.1.x

Client OS: Winows 7 64bit

Server OS of Roaming profile: Windows Server 2008 R2

Operating Systems:

Comments 25 CommentsJump to latest comment

pete_4u2002's picture

is it only happening on win 7 and 2008 machines?

open a support ticket.

 

Moe7's picture

@James007 - we can replicate on all computers.

And thanks for the KB but the only thing I could find was in relation to Windows Defender which is disabled on all our machines.

 

@pete4u2002 - We only run windows 7 clients so haven't been able to test on another OS. In terms of servers, we have tested roaming profiles on Windows Server 2003 for the same result.

 

Will open a support ticket now

.Brian's picture

Best case is to open a ticket here, especially with the latest version

Please click the "Mark as solution" link at bottom left on the post that best answers your question. This will benefit admins looking for a solution to the same problem.

92laslan's picture

We have the same problem on our XenApp 6.5 platform and we use roaming profiles. have tried to uninstall SEP 12.1 and reduced logoff time from 60 sec to 4 sec.

Have support ticket, but no solution yet.

rustychuck's picture

We are experiencing exactly the same problem, and I am pleased (in a way) to find others who are also experiencing the same issue.  Our log off times are any where from 1 minute 30 seconds up to 14 minutes!  We found the problem was happening with 12.1.2 as well as 12.1.3 and so we thought by upgrading to 12.1.4 would solve it but it doesn't.  If we uninstall SEP completely we find our log off times are reduced to 15 - 25 seconds.  

I am waiting to hear back from a Symantec technical support person today so will update further when I know more.

Moe7's picture

Ok I think I have narrowed it down to a permissioning issue on our file servers.

Wireshark was showing that traffic from the client was being denied access to the file server (not exactly sure what SEP is trying to do at this point).

So as a test we created a new share, setup our test account's roaming profile to point to this share and opened the share up to "Everyone" - instant improvement. Log off times went from ~3mins to 30secs.

I then narrowed the access to down domain users/authenticated users, and it was still ok.

Scaling back the permissions to just the test users account and we went back to the long log off times.

 

I believe the issue is that the SEP services on the client are configured to use the local system account which doesn't have read access to the roaming profiles shares.

So I figured we can just configure the services to use a domain account. That's where I hit another road block. I am unable to change the account which the local SEP services use (see screenshot).

Even after turning off the tampering protection and updating the policy on the client I am unable to configure the log on. Obviously also tried stopping the services first (had to do so manually using Start>run>smc -stop) and still no good.

Anyone know how we can configure the log on account for the SEP services running on the local machiine?

 

 

SEP_Services.JPG
Moe7's picture

Anyone able to help with the above? (see screen capture)

.Brian's picture

This is something support will need to troubleshoot via advanced logging and packet captures.

Please click the "Mark as solution" link at bottom left on the post that best answers your question. This will benefit admins looking for a solution to the same problem.

Moe7's picture

Sorry, I was reffering to changing the services log on info.

I am unable to change which account is being used

giesd-ro's picture

Hi,

 

we have the same issue. We have romaing profiles stored on a EMC VNX5300. It takes very long to logoff /writing roaming profile to networkshare). The funny thing is some days it takes up to 5 Minutes to logoff another day only 30 seconds. We always thought its an EMC cifs Problem because after tracing the logoff process the EMC logged many write tries because of write permission failures. The Profile path is like \\server\home\username\profiles. Users have only red permission on the home directory and full permission on username. Today i tried many other things (moving profilepath to other cifs server, to an 2008 R2 server in VMWare or a physical Server) and we had different offline times in every scenario. Today i tested to uninstall Symantec EP 12.1.3... and logoff took only 10 seconds (with EP 3 Minutes). After this i installed EP 12.1.4... and it also took only 10 seconds to logoff. I will udate all client to 12.1.4... and hope the Problem is fixed then.

Changelog from 12.1.4 has one fix pointing to this problem

Servers are slow or unresponsive
Fix ID: 3241493
Symptom: After you install the Symantec Endpoint Protection client without Network Threat Protection, the file share server appears to be offline, or becomes extremely slow and unresponsive.
Solution: Fixed a condition that lead to a deadlock in the cache manager.
 
I hope i was able to help. I also help this was fixex our problem ;-)
 
Best
Robert
AjinBabu's picture

Hi, 

Please log a case with support for advanced trouble shooting.

Regards

Ajin

rustychuck's picture

Following on from my previous comment, I finished my call with a Symantec Engineer.  It turns out that there is an issue with the current SEP client with the included BASH driver.  The BASH driver has been known to cause problems for particular environments.  Like the SEP client, the BASH driver has definitions which can be applied to it.  Apparently, Symantec are currently working on new BASH drivers which help prevent this issue from happening.  The workaround for us, was to disable the BASH driver from starting which has dramatically helped.  Preventing the BASH drivers though seems to affect the Auto-Protect feature so we will wait for the next update to SEP.

We also found that we have the group policy 'Delete cached copies of roaming profiles' disabled.  Group Policy was trying to remove the cached profile at log off but couldn't because another process was holding the profile open.  Disabling this GP setting has also reduced our log off times as well.  Instead we use the policy setting 'Delete user profiles older than a specified number of days on system restart' and set this to 3 days.

Where machines were taking 8 - 10 minutes to log off, with both of these fixes, machines now take around 15 seconds to log off which is amazing.  I think Symantec need to release an update for the SEP client pretty sharpish because, going by the number of people in this forum who are experiencing the same issue, this is becoming a big problem!  We need to push Symantec for an emergency release.....

92laslan's picture

Hi,

my support case is also closed with the workaround disable BHDrvx64.sys

Symantec support explain it will then reverting back to the September 24 2013 definitions (using driver version 7.8.0.10)

Symantec software team is working on to find the exact root cause for this known issue. so it may be a while before the new BASH 8.2 definitions containing the fix will be released

I have been guaranteed that new definition will work on the old driver.
Tested and now I use 6 sec to log off :-)

To disable BHDrvx64.sys

Open cmd
Run command "sc config BHDrvx64 start = disabled"
Reboot the machine

Regards,

Lasse

 

 

 

.Brian's picture

92laslan,

Excellent info. Thanks for sharing this.

Please click the "Mark as solution" link at bottom left on the post that best answers your question. This will benefit admins looking for a solution to the same problem.

Moe7's picture

Thanks you 92laslan and rustychuck. I haven't had much luck with symantec support yet.

Did they advise when they will be rolling back to the previous driver in a definition update? Or do we have to disable the driver until they release a fix?

flutti's picture

Same issue here, we've been experiencing this since every SONAR definitions release newer than 09/24/2013 r11. Luckily found it out by ourselves shortly after the release of the 10/24/2013 r11 definitions.

Symantec case has been opened on October 31st, still no solution whatsoever.

Making a too long story short, we're using backdated SONAR definitions ever since October 26th.

RoPe's picture

We are also seeing the same issue as well.  Have had an open case with Symantec since early November with no resolution yet other than disabling BASH or using old defs, which we do not view as good workarounds for a large amount of client computers. 

One strange behavior I have noticed is if we trigger the UAC mechanism (Windows 7 computer) before logging off (in this case the user is a local admin), the logoff happens very quickly in most cases.  For example, if we open either the SEP client or a CMD prompt in admin mode, the UAC comes up and we hit OK.  Once that happens, logging off is fast.  This is not a fix, just something I noticed in our case. 

Moe7's picture

Is there anyway way to use an old definitions wihtout using a LiveUpdate server?

.Brian's picture

Try this

http://www.symantec.com/docs/TECH102935

Please click the "Mark as solution" link at bottom left on the post that best answers your question. This will benefit admins looking for a solution to the same problem.

Moe7's picture

Thanks, that's exactly what I was after!

.Brian's picture

Good deal, hopefully that works...

Please click the "Mark as solution" link at bottom left on the post that best answers your question. This will benefit admins looking for a solution to the same problem.

mart-23's picture

I wish I searched earlier for this post before my own troubleshooting. It's been on my list of things to fix for the last couple of months about slow logins with roaming profiles.

I also tracked it down to the BHDrvx64.sys driver. Having disabled it, logouts was fast again. When I re-enabled it, log outs were still fast! I then realised that I had set the service to start automatic rather than system (as is the default)

sc config BHDrvx64 start= auto

or

sc config BHDrvx86 start= auto     (for 32bit machines)

 

I'm testing this on a several PC's now. At least the service is not disabled.

 

Regards

 

Radoslaw Ambrozkiewicz's picture

Hello,

I had the same problem.

Today from Symantec Technical Support I get a new Proactive Threat Protection defintions date 18.12.2013 r11 for testing and seems that all problems go away.
It will be official released next week

Regards,

 

.Brian's picture

Moe,

Have you tested with the new PTP def set? Did it fix it?

Please click the "Mark as solution" link at bottom left on the post that best answers your question. This will benefit admins looking for a solution to the same problem.