Endpoint Protection

 View Only
  • 1.  Runaway Upgrade Processes (obscene bandwidth consumption)

    Posted Jan 04, 2012 03:46 PM

    So our network team recently noticed an interesting trend in our network throughput from our AV server.  They noticed that our servers where generating a combined, and sustained, throughput of about 500 Mbps between 1am and 7am.

    Well that time frame matched an upgrade schedule that we performed quite a while back.  But that didn't seem right since almost all of the clients completed their upgrades a long time ago.  We have left the schedule in place to catch a very small percentage of straggling clients.  However when looking at this more deeply it appears that a small handful of machine (about 30-35) are in a very problematic state.  They can't seem to complete the upgrade and it is failing for some reason (we are still trying to determine why).  But instead of failing constructively and letting us know they have done so, they just seem to keep trying over, and over and over, etc.

    By the end of each 6 hour window, each of these clients has downloaded between 35 and 40 GB of data (install package data).  Yes, that is not a typo, I did mean to say 35 to 40 Gigabytes.  Between these 30 some odd machines they are pulling a total of about 1.3 Terabytes of data from my servers.  And apparently they have been doing this nightly for quite a while.  Thank goodness this is all over a LAN.

    Have anyone else seen this and if so, where you ever able to determine the cause and engineer a fix?

    Thanks for your time.



  • 2.  RE: Runaway Upgrade Processes (obscene bandwidth consumption)

    Broadcom Employee
    Posted Jan 04, 2012 11:16 PM

    The figures seems to be on higher side.

    What is the version of SEP ? Did you check if delta package being upgraded?



  • 3.  RE: Runaway Upgrade Processes (obscene bandwidth consumption)

    Posted Jan 05, 2012 04:21 AM

    First, you would need to identify what traffic is this....

     

    Use a network monitor tool and check on what port is the traffic getting generated on .



  • 4.  RE: Runaway Upgrade Processes (obscene bandwidth consumption)

    Posted Jan 05, 2012 04:28 AM

    You could use Wireshark and check the traffic.



  • 5.  RE: Runaway Upgrade Processes (obscene bandwidth consumption)

    Posted Jan 05, 2012 05:00 AM

    I would try exporting the package you are deploying from the SEPM and try installing it directly on one of the problem clients and see if this generates any error's you can see during the install/upgrade.

    Do you have a proxy server configured on the SYSTEM account on the machines that have the problems??  This can cause the traffic data to be malformed and the client machine to reject the data over and over again.

    Check the following two registry keys to see if this is the case...

    HKEY_USERS\.DEFAULT\Software\Microsoft\Windows\CurrentVersion\Internet Settings\ProxyEnable 

    This should be set to 0 unless you have sofwtare that specifically requires the System account to have a Proxy Configured.


    HKEY_USERS\.DEFAULT\Software\Microsoft\Windows\CurrentVersion\Internet Settings\Connections\DefaultConnectionSettings   

    This should be deleted if you have set it to be 0 above.

    Reboot the machine after changing the above then see if it updates ok.



  • 6.  RE: Runaway Upgrade Processes (obscene bandwidth consumption)

    Posted Jan 10, 2012 05:57 PM

    I'll try the package idea and see if it gives us any clues.  It's not a proxy issue (none of the machines we have looked at so far have had a proxy configured).



  • 7.  RE: Runaway Upgrade Processes (obscene bandwidth consumption)

    Posted Mar 05, 2012 02:30 PM

    What version of SEP 11 are you running and what is the OS of the problematic machines?  32 or 64 bit?

    Thanks.



  • 8.  RE: Runaway Upgrade Processes (obscene bandwidth consumption)

    Posted Mar 05, 2012 07:46 PM

    You should also check the temp folders: user/temp and windows/temp for any installers.

    They could probably be cleaned using Windows Disk Cleanup.



  • 9.  RE: Runaway Upgrade Processes (obscene bandwidth consumption)

    Posted Mar 06, 2012 06:49 AM

    I would appreciate your working with support on a case, so that if a product defect exists for this use case, it can either be identified as a duplicate or entered and fixed.