I assume you don't have any GUPs in place?

How long were they off and were clients up to date before that?

No, we don't have any GUPs.

Hi,

When you power on the machine after a gap of 1 or 2 weeks it’s possible that the SEPM console does not have the latest virus definitions and thereby the clients won’t have it too. The best strategy in such cases would be to upgrade the SEPM console with the latest virus definitions first. As far as the SEP clients are concerned the latest rapid release definitions should help.

The rapid release definitions can be downloaded and kept at a centrally shared location so that the clients can download that exe file(If possible may be create a script file so that the exe file is installed when the computer starts and thereby the AV/AS definitions which consume bandwidth can be updated before they contact their respective SEPM) and update their definitions, by doing so it would reduce the traffic in the network between the SEP clients and the SEPM, because when the SEP clients contact the SEPM, the SEPM checks with its own database for the version of definitions available and if the SEP client has the latest or a day old definitions, it distributes the updates which are a few KB’s in size, however if the SEP client has definitions which are a week or two old, then the SEPM will dispatch the FULL.ZIP file and the size can be around 50 to 70 MB’s(Approximate value, it may vary) which will consume a lot of bandwidth.

Reference Article: Managing SEPM & SEP after vacation

http://www.symantec.com/connect/articles/managing-...

Update: Here is my plan to address the situation:

1. Unplug the network wire from virus server 1.  Power it on and access the console locally.  (This will allow us to make changes, but not allow the clients to update.)
2. Modify the “LiveUpdate Content Policy” for all machines.  (This setting defines which definitions the clients download and use.)
   1. Currently, it is set to “Latest”…we’ll set it to a static definition set that matches the latest definitions that our clients had before the outage.  Let’s refer to that as the 10/29 definition (I need to confirm the day).  This forces all clients to run the 10/29 definition, which most of them already have and will not need to download.
3. Modify the Randomization setting for definition updates.  It’s currently set to something like 15 minutes.  We can change it to 24 hours.  When the clients check in, they will generate a random time between 0 and 24 hours, and will reconnect at that time to download the definitions.
4. Plug the network wire back into virus server 1. 
5. Power on virus server 2.  It will automatically update itself to match virus server 2.
6. Now all of the machines will start checking in, but they won’t download the definition updates, so the bandwidth won’t be a problem.  They’ll pick up their randomization setting change and wait until later.
   1. Any clients that have a definition older than 10/29 (but within 15 revisions) will download the microdef to update to the 10/29 definition.  (A small download, which will be randomized across 24 hours.
   2. Clients with definitions older than about 10/24 will have to download the full definition set (almost 200MB now). That was always the case, even before Sandy, because they are outside the 5 day window.  At least it will be across the large randomization window.
7. We let everything sit for about 2 days.  That ensures:
   1. All clients are on the new randomization schedule.
   2. All clients are up to 10/29 definition.
   3. The SEP Server has a chance to download the latest (Let’s call it 11/6) definition and calculate the delta between 11/6 and 10/29.  The fact that our server has been offline actually helps us here because the server will consider this gap to be only a single definition.  (as if we were frozen in time)  The server will create the delta, which will be larger than the average delta (200-300KB), but still reasonable.  Expect it to be around 2 MB.  That is what the majority of the SEP clients will end up downloading.
8. Go back and change the “LiveUpdate Content Policy” to latest.
   1. Clients will begin downloading the 2MB microdef to update to the 11/6 definition.
   2. Clients will do this downloading across the 24 hour randomization schedule.
   3. At this point, we monitor the network.  If anything becomes overloaded, we have the option of backing out by powering down the SEP servers and rethinking our approach.

One of our concerns is that if we follow this plan, is it possible for the SEP clients to start grabbing new definitions before receiving the “LiveUpdate Content Policy” change that tells them to hold onto the old definitions and not update?

Or will the clients definitely communicate to the server and receive that setting before they attempt to download the definitions?

Any thoughts on the order of how the SEP client connects and downloads policy and definitions?

Don't forget that you can use the built in IIS throttling as well to take a bit of the load off of the network.  IIS should let you limit both bandwidth and number of connections.

How to throttle network bandwidth used by the Endpoint Protection Manager (SEPM) website in Microsoft's Internet Information Server (IIS)

http://www.symantec.com/docs/TECH104518

Hi,

I believe your approach is correct.

One of our concerns is that if we follow this plan, is it possible for the SEP clients to start grabbing new definitions before receiving the “LiveUpdate Content Policy” change that tells them to hold onto the old definitions and not update?

--> Every time clients will first connect to SEPM, will download index file etc...& then start downloading definitions.

Clients will received the new policy first  and according to the policy update they will start downloading definitions.

Hi,

Check this article as well.

Symantec Endpoint Protection 11.x LiveUpdate "Micro Definition" Updates Explained

http://www.symantec.com/docs/TECH180196

Thanks.  At which step in this article is the SEP client downloading and applying the changes to policy settings?

http://www.symantec.com/business/support/index?page=content&id=TECH191617

I am sure you won't have a problem with bandwidth. After connecting the SEPM back, it will download the lates revision only. So the SEPM will have 1 new revision and 14 very old revisions. When the clients connects to SEPM, the clients already have one of the 14 very old revision and they need to be updated to the latest. So, a delta will be created, let's say 5-10MB , not a full definition. And if you have randomization and not all clients are turned on at the same time, the bandwidth will survive.