Video Screencast Help

Symantec endpoint protection exceptions (profile corruptions)

Created: 03 Dec 2013 • Updated: 03 Mar 2014 | 15 comments
This issue has been solved. See solution.

Hi,

When a centralized exception (extension) is added in SEPM with action 'ignore' should SEP on clients skip scanning the filetype altogether or simply ignore alerts if a file is infected?

We've specified a centralized exception for the extension 'DAT' on our networks. Looking at a client computer registry I can see the rule is present yet strangely when doing any AV scan RTVScan.exe also opens all users ntuser.dat. By any AV scan I actually mean it, if I scan 'C:\temp' folder (which contains only 1 desktop.ini) sysinternals process monitor shows RTVScan accessing the profilelist registry key and then scanning all user ntuser.dat's.

 

There's a wider issue this is causing on our networks related to corrupt profiles. Like all windows networks with roaming profiles we occasionally get profile corruptions; and if particularly bad windows attempts to recover the ntuser.dat on the client machine, a process which is often accompanied by an event viewer entry:

EventID: 5

{Registry Hive Recovered} Registry hive (file): '\??\C:\Users\UserAccount\ntuser.dat' was corrupted and it has been recovered. Some data might have been lost.

We resolve these corruptions by resetting/restoring a users network ntuser.dat back to default. In most cases after restoring ntuser.dat the user will then successfully log on as windows can see the network profile is newer than the local corrupted profile on the machine. What we're finding is that when RTVScan accesses the ntuser.dat on local machines it triggers windows to rebuild these corrupt profiles again which in turn updates the modified date on them. On next logon the locally cached copy is seen as newer and is used instead of the fresh network copy. As an AV scan is triggered each time new definitions are loaded these profiles can be rebuilt multiple times throughout the day making it hard to stay on top of corruptions, especially as windows aged profile deletion will never see them as being old enough to remove.

 

SEPM Version: 11.0.6100.645

Client SEP Version: 11.0.6100.645

 

Thank you in advance for any help with this matter.

Andrew

Operating Systems:

Comments 15 CommentsJump to latest comment

.Brian's picture

My understanding is that when an exception is added it will be ignored so in this case ntuser.dat should not be scanned. Is this happening during every scan (auto-protect and scheduled)? Did you try excluding .dat file extension instead?

 

Please click the "Mark as solution" link at bottom left on the post that best answers your question. This will benefit admins looking for a solution to the same problem.

JohnAF's picture

Thanks for the quick replies.

James, the post seem's to suggest it's only possible to block scanning of ntuser.dat by adding an exception for the .DAT extensions. My issue is that I've added DAT in as an exception but Rtvscan.exe seems to ignore the rule.

 

Brian, I'm not sure if this occurs during an on-access scan (I'll have to test), but it seems both manual and scheduled scan's cause Rtvscan to open ntuser.dat on the clients, this includes manually created scans designed to target folders external to the users folder.

I have screenshots of this happening if it makes things easier, process monitor seems to show RTVScan performing a writefile operation on ntuser.dat among lots of createfile etc.

 

Thanks

.Brian's picture

Have you seen this article with Procmon and how to configure to see auto-protect events?

http://www.symantec.com/docs/TECH98079

Please click the "Mark as solution" link at bottom left on the post that best answers your question. This will benefit admins looking for a solution to the same problem.

JohnAF's picture

Thanks Brian,

Unfortunately I don't know enough about minifilter driver monitoring to be 100% sure. I've made the altitude changes and can see SRTSP.sys on the system stack during call's to query information on the ntuser.dat file, but I don't see any evidence of SRTSP IRP operations.

I assume auto-protect and scheduled scans should use the same exceptions?

 

Going back to Rtvscan, I've attached a screenshot from process monitor filtered for Rtvscan writefile operation. The scan was of C:\temp only (no memory, common or well known locations), the folder contained a desktop.ini and test.dat. I can see Rtvscan scanning the temp folder and files towards the end of procmons logs (no writefile operation takes place against them), but above that the vast majority of Rtvscans work seems to be in polling all user profiles on the system and scanning them.

Even if I didn't have a valid exception for .dat files (which I believe I do) I fail to understand why SEP chooses to scan all ntuser.dat's in this situation, am I missing something obvious?

 

Thanks again,

Andrew

RtvscanDats.png
.Brian's picture

They use the same exception if you specify it.

There are 3 options to exclude:

  1. Auto-Protect
  2. Scheduled and On-demand
  3. All

 

Please click the "Mark as solution" link at bottom left on the post that best answers your question. This will benefit admins looking for a solution to the same problem.

JohnAF's picture

We have a centralized exception policy applied to all clients which includes the following:

Exception item: DAT,VCD etc
Exception type: Security Risk Extensions
Action: Ignore

It appears this should apply to all scans for these machines.

 

 

 

 

.Brian's picture

On that screen where you add the extension, it will also allow you to choose from the 3 I mentioned above.

Please click the "Mark as solution" link at bottom left on the post that best answers your question. This will benefit admins looking for a solution to the same problem.

JohnAF's picture

When adding extensions SEPM doesn't allow you to input a dot prefix, it requires only the extension without the dot.

JohnAF's picture

I've looked into this a little further; following is part of the output from a SEP debug log after scanning C:\Temp:

 

11:32:04.102794[_2132]|Started scanning a new scan item: C:\Temp.
11:32:04.560912[_2132]|Processing directory 'c:\Temp'.
11:32:04.566807[_2132]|Processing file 'c:\Temp\desktop.ini'.
11:32:04.582642[_2132]|Processing file 'c:\Temp\test.dat'.
11:32:04.582843[_2132]|CSavScanSink::SafeOnNewFile2 - Excluding file - c.
11:32:04.582984[_2132]|Processing file 'c:\Temp\Test.mdf'.
11:32:04.583125[_2132]|CSavScanSink::SafeOnNewFile2 - Excluding file - c.
11:32:04.583265[_2132]|Processing file 'c:\Temp\Test.VCD'.
11:32:04.583406[_2132]|CSavScanSink::SafeOnNewFile2 - Excluding file - c.
11:32:05.595559[_2132]|Done Status: completed.

 

So it seems the scan itself is identifying and correctly excluding .dat files, but before the scan begins there's some kind of check performed that searches each user profile on the system, this check ignores the .dat exclusion. I'm thinking maybe this check isn't actually scanning the ntuser.dat at all but instead retrieving data from it or writing an entry to it.

Is there someone with developer knowledge who can explain what I'm seeing here?

 

Thanks,

 

Edit:

API monitor shows Rtvscan.exe loads all user hives and querys values in Software\Symantec\Symantec Endpoint Protection\AV. Is there a way to prevent this behaviour or is it needed? As mentioned above it's triggering windows into attempting a recovery of corrupted user hives.

.Brian's picture

You're best bet here would be to open a support ticket with Symantec.

Please click the "Mark as solution" link at bottom left on the post that best answers your question. This will benefit admins looking for a solution to the same problem.

.Brian's picture

Do you need more assistance with your problem or were you able to get it resolved?

If you could post an update for followers of this thread that would be most helpful.

Otherwise, if resolved, you can close the thread out by clicking the "Mark as solution" link at the bottom left on the most helpful post. If multiple posts helped to solve your problem, please click the "Request split solution" link at the bottom left, select the most helpful posts and click the "Submit" button. This will benefit admins looking for a resolution to the same problem.

Thanks and take care,
Brian

Please click the "Mark as solution" link at bottom left on the post that best answers your question. This will benefit admins looking for a solution to the same problem.

JohnAF's picture

Hi Brian,

Apologies for not getting back. We did get to the bottom of this in the end with the help of Symantec Technical Support and the replies here.

Access of the NTUser.dat files was caused by a function in RTVScan which regularly checks all user defined scans in each users profile. As previously mentioned this function happens regardless of exceptions as it's not part of a scan.

Registry changes can be used to prevent RTVScan from checking user defined scan's which are detailed below. It's important to note that SEP12.1 changed the storage location of user scheduled scans out of the user registry and so doesn't have this issue.

The following was taken from an internal document I believe:

--------------

Cause

In the "MainTimer" function, RtvScan will regularlly check all users defined schedule scan which will hive related "ntuser.dat". This operation will consume CPU. As this server has several login user accounts, so this process will take quite long time and keep CPU high.
 

Solution

Try this workaround for SEP 11.0 RU6 or higher:

1. Add a “ScanStartupDelay” register key under
[HKEY_LOCAL_MACHINE\SOFTWARE\Symantec\Symantec Endpoint Protection\AV\ProductControl], "ScanStartupDelay"=dword:7776000
 
2. Add a “DisableRTScheduledScanUpdate” register key under
[HKEY_LOCAL_MACHINE\SOFTWARE\Symantec\Symantec Endpoint Protection\AV\ProductControl], " DisableRTScheduledScanUpdate "=dword: 00000001
 
3. Add a “ReloadRTScheduledScanHours” register key under
[HKEY_LOCAL_MACHINE\SOFTWARE\Symantec\Symantec Endpoint Protection\AV\ProductControl], " ReloadRTScheduledScanHours "=dword: 00002160
 
(OBS: The values should be entered in decimal format. Also, the preceding zeroes on the numbers may be automatically truncated by the Registry Editor.)
 
4. Restart the rtvscan service or reboot the computer .

The same issue does not happen in SEP 12.1.Please update me once you have checked it so that we will close this case.

--------------

 

After getting this response and searching for specific terms from it I also found the following post which mentions the same issue and relates to the CPU usage:

https://www-secure.symantec.com/connect/ideas/ability-stop-sep-scanning-user-profiles-user-defined-scans-servers

 

I've tested the registry changes and they do appear to fix the issue, and so will act as a workaround until we upgrade SEP.

 

Thank's for all the replies here. The help/guidance was greatly appreciated.

SOLUTION