Troubleshooting high bandwidth utilization issues with SEPM & SEP clients.
One of the great things with advances in antivirus definition updates is the use of Delta or xDelta updates. Basically these updates are much smaller than downloading the full definition file for a client and conserving bandwidth. But how do you know your clients are downloading the Full or xDelta definition packages? To my knowledge the SEP client logs do not track which updates are downloaded from the management server. Here is a quick document which can help you determine (and correct the issue) if your SEP clients are downloading the full definitions.
Tools you will need for this troubleshooting session:
Install WireShark on your management server. Open WireShark. Click on Capture, Interfaces and click start on the Interface with the most packets detected.
Let the packet capture run for approximately 15 – 20 minutes, depending upon the amount of traffic you are seeing. Save the packet capture and copy to your local system that has LogParser installed.
Open LogParser. The command you want will be similar to the following:
Logparser “SELECT text INTO C:\temp\output.log FROM C:\temp\packetcapture.pcap WHERE text LIKE ‘%C60DC234-65F9-4674-94AE-62158EFCA433%’” –o:CSV
This command is assuming you saved your packet capture file as packetcapture.pcap and copied it to your local system in C:\temp. The C60DC234… is from the following location on the management server: c:\Program Files\Symantec\Symantec Endpoint Protection Manager\Inetpub\content\ContentInfo.txt. This list contains the various definitions that the SEPM has stored and available for SEP clients. In my case the C60DC234… is for the 32-bit version antivirus definition files.
Now go back to your SEPM server (not the console, but the physical server). Open the location F:\Program Files\Symantec\Symantec Endpoint Protection Manager\Inetpub\content\{C60DC234-65F9-4674-94AE-62158EFCA433}\90806023\
Inside this folder you should see a directory called Full, a full.zip file, and some xdelta####.dax files. These dax files are your incremental updates for SEP.
If you only have three .dax files many of your clients are likely downloading the full.zip file, which is normally over 50 MB. Have a couple thousand clients in pull mode getting that update at the same time… it can crush a robust network. From what I have seen, Symantec normally releases three revisions per day. If you server is setup to check every 6 – 8 hours for updates, you will get all three updates. So right there you have three revisions. If your SEP clients are only updating once per day it adds more complexity to the situation. Now, your clients are actually going to need the full.zip the next morning (depending upon when they last updated) because the server may have already grabbed the latest definition files. This puts the clients more than 3 revisions behind what is stored on the server forcing the client to download the full definition file. Think about Monday morning. Every client (if offline all weekend) will need the full definition file. Save yourself the trouble and kick the number of revisions stored on the SEPM server to more than 16.
To increase the number of revisions stored in your SEPM infrastructure, follow these steps. Open your SEPM console and login, as a system administrator. Click the Admin item on the left tool panel. Click the Servers, Local Site, Edit Site Properties. (see below graphic)
When the Site Properties window opens look to the bottom of the screen (see below screen shot). Increase the number of revisions and the check the check box. Save your settings and continue to monitor updates during the next couple of days. It may take up to three days before your network recovers from bandwidth issues. Also make sure your SEPM servers have sufficient space to store all these revisions. Our instance (at 16 revisions) consuming just over 5 GB of data.
