Video Screencast Help

Turning SEPM servers back on

Created: 05 Nov 2012 | 16 comments

I’m looking for some advice.  We have a SEP 11 environment with about 8000 machines on it.  We had our revisions set to 15, which gives us about 5 days worth of definitions.  During the recent storm, we needed to keep as much bandwidth available as possible, so we turned off the SEPM servers to stop the clients from getting antivirus updates to save bandwidth.

 

Now we are that point where we need to turn the servers back on, but we are worried about all of the clients calling into the server and downloading a full definition set and crippling our network.  What is the best way to handle this situation?

Comments 16 CommentsJump to latest comment

_Brian's picture

I assume you don't have any GUPs in place?

How long were they off and were clients up to date before that?

Please click the "Mark as solution" link at bottom left on the post that best answers your question. This will benefit admins looking for a solution to the same problem.

_Brian's picture

How long were they off and were clients previously up to date before turning off?

What is your randomisation set to?

Randomizing content downloads from the default management server or a Group Update Provider

https://www.symantec.com/business/support/index?pa...

Please click the "Mark as solution" link at bottom left on the post that best answers your question. This will benefit admins looking for a solution to the same problem.

Chetan Savade's picture

Hi,

When you power on the machine after a gap of 1 or 2 weeks it’s possible that the SEPM console does not have the latest virus definitions and thereby the clients won’t have it too. The best strategy in such cases would be to upgrade the SEPM console with the latest virus definitions first. As far as the SEP clients are concerned the latest rapid release definitions should help.

The rapid release definitions can be downloaded and kept at a centrally shared location so that the clients can download that exe file(If possible may be create a script file so that the exe file is installed when the computer starts and thereby the AV/AS definitions which consume bandwidth can be updated before they contact their respective SEPM) and update their definitions, by doing so it would reduce the traffic in the network between the SEP clients and the SEPM, because when the SEP clients contact the SEPM, the SEPM checks with its own database for the version of definitions available and if the SEP client has the latest or a day old definitions, it distributes the updates which are a few KB’s in size, however if the SEP client has definitions which are a week or two old, then the SEPM will dispatch the FULL.ZIP file and the size can be around 50 to 70 MB’s(Approximate value, it may vary) which will consume a lot of bandwidth.

Reference Article: Managing SEPM & SEP after vacation

http://www.symantec.com/connect/articles/managing-...

Chetan Savade
Technical Support Engineer, Endpoint Security
Enterprise Technical Support
CCNA | CCNP | MCSE | SCTS |

Don't forget to mark your thread as 'SOLVED' with the answer that best helps you.<

ed16's picture

Update: Here is my plan to address the situation:

  1. Unplug the network wire from virus server 1.  Power it on and access the console locally.  (This will allow us to make changes, but not allow the clients to update.)
  2. Modify the “LiveUpdate Content Policy” for all machines.  (This setting defines which definitions the clients download and use.)
    1. Currently, it is set to “Latest”…we’ll set it to a static definition set that matches the latest definitions that our clients had before the outage.  Let’s refer to that as the 10/29 definition (I need to confirm the day).  This forces all clients to run the 10/29 definition, which most of them already have and will not need to download.
  3. Modify the Randomization setting for definition updates.  It’s currently set to something like 15 minutes.  We can change it to 24 hours.  When the clients check in, they will generate a random time between 0 and 24 hours, and will reconnect at that time to download the definitions.
  4. Plug the network wire back into virus server 1. 
  5. Power on virus server 2.  It will automatically update itself to match virus server 2.
  6. Now all of the machines will start checking in, but they won’t download the definition updates, so the bandwidth won’t be a problem.  They’ll pick up their randomization setting change and wait until later.
    1. Any clients that have a definition older than 10/29 (but within 15 revisions) will download the microdef to update to the 10/29 definition.  (A small download, which will be randomized across 24 hours.
    2. Clients with definitions older than about 10/24 will have to download the full definition set (almost 200MB now). That was always the case, even before Sandy, because they are outside the 5 day window.  At least it will be across the large randomization window.
  7. We let everything sit for about 2 days.  That ensures:
    1. All clients are on the new randomization schedule.
    2. All clients are up to 10/29 definition.
    3. The SEP Server has a chance to download the latest (Let’s call it 11/6) definition and calculate the delta between 11/6 and 10/29.  The fact that our server has been offline actually helps us here because the server will consider this gap to be only a single definition.  (as if we were frozen in time)  The server will create the delta, which will be larger than the average delta (200-300KB), but still reasonable.  Expect it to be around 2 MB.  That is what the majority of the SEP clients will end up downloading.
  8. Go back and change the “LiveUpdate Content Policy” to latest.
    1. Clients will begin downloading the 2MB microdef to update to the 11/6 definition.
    2. Clients will do this downloading across the 24 hour randomization schedule.
    3. At this point, we monitor the network.  If anything becomes overloaded, we have the option of backing out by powering down the SEP servers and rethinking our approach.

 

One of our concerns is that if we follow this plan, is it possible for the SEP clients to start grabbing new definitions before receiving the “LiveUpdate Content Policy” change that tells them to hold onto the old definitions and not update?

 

Or will the clients definitely communicate to the server and receive that setting before they attempt to download the definitions?

_Brian's picture

When the clients check in, they should received latest policy updates. I've never tried this before but in theory it should work.

Please click the "Mark as solution" link at bottom left on the post that best answers your question. This will benefit admins looking for a solution to the same problem.

ed16's picture

Any thoughts on the order of how the SEP client connects and downloads policy and definitions?

_DW's picture

Don't forget that you can use the built in IIS throttling as well to take a bit of the load off of the network.  IIS should let you limit both bandwidth and number of connections.

 

 

How to throttle network bandwidth used by the Endpoint Protection Manager (SEPM) website in Microsoft's Internet Information Server (IIS)

http://www.symantec.com/docs/TECH104518

Chetan Savade's picture

Hi,

I believe your approach is correct.

One of our concerns is that if we follow this plan, is it possible for the SEP clients to start grabbing new definitions before receiving the “LiveUpdate Content Policy” change that tells them to hold onto the old definitions and not update?

--> Every time clients will first connect to SEPM, will download index file etc...& then start downloading definitions.

Clients will received the new policy first  and according to the policy update they will start downloading definitions.

Chetan Savade
Technical Support Engineer, Endpoint Security
Enterprise Technical Support
CCNA | CCNP | MCSE | SCTS |

Don't forget to mark your thread as 'SOLVED' with the answer that best helps you.<

ed16's picture

Thanks.  Are there any articles that outline the check-on process followed, that would should that the policy settings are applied before the client decides to start downloading definitions?

Chetan Savade's picture

Hi,

This article should help.

Symantec Endpoint Protection: The Heartbeat Process

http://www.symantec.com/docs/TECH191617

Chetan Savade
Technical Support Engineer, Endpoint Security
Enterprise Technical Support
CCNA | CCNP | MCSE | SCTS |

Don't forget to mark your thread as 'SOLVED' with the answer that best helps you.<

Chetan Savade's picture

Hi,

Check this article as well.

Symantec Endpoint Protection 11.x LiveUpdate "Micro Definition" Updates Explained

http://www.symantec.com/docs/TECH180196

Chetan Savade
Technical Support Engineer, Endpoint Security
Enterprise Technical Support
CCNA | CCNP | MCSE | SCTS |

Don't forget to mark your thread as 'SOLVED' with the answer that best helps you.<

ed16's picture

Thanks.  At which step in this article is the SEP client downloading and applying the changes to policy settings?

http://www.symantec.com/business/support/index?page=content&id=TECH191617

Chetan Savade's picture

Hi,

Check the third step.

3. SEP client performs an HTTP GET of index.dat from the SEPM and compares it against the client copy for any deltas.

  • Content differences will check against LiveUpdate policy for current location.

Chetan Savade
Technical Support Engineer, Endpoint Security
Enterprise Technical Support
CCNA | CCNP | MCSE | SCTS |

Don't forget to mark your thread as 'SOLVED' with the answer that best helps you.<

Chetan Savade's picture

HI Ed16,

Is there any update on this issue?

If issue is resolved then don't forget to mark your thread as 'SOLVED' with the answer that best helps you.

Chetan Savade
Technical Support Engineer, Endpoint Security
Enterprise Technical Support
CCNA | CCNP | MCSE | SCTS |

Don't forget to mark your thread as 'SOLVED' with the answer that best helps you.<

Yahya's picture

I am sure you won't have a problem with bandwidth. After connecting the SEPM back, it will download the lates revision only. So the SEPM will have 1 new revision and 14 very old revisions. When the clients connects to SEPM, the clients already have one of the 14 very old revision and they need to be updated to the latest. So, a delta will be created, let's say 5-10MB , not a full definition. And if you have randomization and not all clients are turned on at the same time, the bandwidth will survive.