Endpoint Protection

Does anyone else experience the random freezing of VMs referenced in the following article:

https://support.symantec.com/en_US/article.TECH236543.html

We are seeing these random freezing issues happen on our VMs and, though we have implemented the workarounds for our 14 MP1 SEP clients it hasn't fixed the issue. Does fully upgrading to 14 MP2 resolve this? Am I the only one experiencing this issue?

Per 14 MP2 fix notes, this was fixed:

http://www.symantec.com/docs/INFO4375

SEP 14 MP1 causes random hangs on servers and desktops

FIX ID: 4071163, 4071509, 4075683, 4079555, 4080313, 4082812, 4083960, 4085244

Symptom: Servers and desktop computers lock up or hang after they upgrade to Symantec Endpoint Protection 14 MP1. During this hang, you cannot connect remotely to the systems using programs such as Remote Desktop Connection. This issue also occurs with virtualized servers and desktop computers hosted on VMware and Hyper-V. You do not get a log on screen, nor can you access anything on the computers while they are in this state, though the affected systems may respond to pings. This issue appears to occur at random. A restart allows the affected computers to work again, but the issue may return.

Solution: Changed the Auto-Protect code so that it does not encounter deadlocks under certain circumstances.

We were facing the same issue and as per symantec's sugestion upgraded the AV to 14 MP2. But the issue is still poping up on random serves.

Hi,

 

We are also facing the same issue with version 14 MP2. This morning a server and a few Windows 7 clients freezed.

Needed to restart in command prompt and perform CHKDSK /F to start the clients.

 

After CHKDSK, we managed to login to Windows but SEP service failed to start.

Upon reinstallation of SEP, the client again freezed upon a restart.

 

I hope Symantec support would reply this thread ASAP.

We are also seeing these issues with the MP2 client on what seems to be primarily 2K8 VM servers

Also here. I have 63 Windows 2008 R2 running XenDesktop 7.8 on XenServer. Several of the servers goes dead after a few minutes after the server gor definition updates.

Any solution symantec?

 

I have tried uninstalling (with deletion of Symantec folders in Programdata, Program Files and Program Files\Common Files) and reinstalling Symantec Endpoint client to affected computers. However, computer freezes again upon a restart. Please provide some feedback soonest. 

Hello YL Lam, andymease, and dynamitten,

Thanks for reporting this.  We are investigating now: there are several reports of freezes for Windows 2008 and Windows 7 computers running SEP 14.  (SEP 12.1 is not affected.)

It is not yet confirmed, but the reported freezes coincide with a new SDS 1.4.1 Engine release.  A change made at the SEPM can prevent the freeze.  I recommend that the definitions for any SEP 14 clients be set to an earlier set of definitions than the release that contained the new SDS Engine (20170802.008).  Please choose definitions 20170802.002.  Here are instructions: 

 

How to Backdate Virus Definitions in Symantec Endpoint Protection Manager
http://www.symantec.com/docs/TECH102935

 

I will update this thread later with additional new information, as required.

Many thanks - please try setting the SEP 14 definitions to the final release before the new SDS Engine and update this thread with your progress!

A dedicated article on this subject:

Hang at logon following the August 2, 2017 SDS engine update
http://www.symantec.com/docs/TECH247190

 

Attached is a screenshot of my SEPM definition list. Could you tell me which specific definition to choose?

After the latest definition update to Symantec Endpoint 14 Windows stuck. I have this problem on two 32 bit Windows 7.

SEP 14 is affected: SEP 12.1 is not.  Are your Win7 and Win2008 machiens using 32-bit or 64?  Are they using reduced size definitions or not?

Ideally, backdate the definitons as precisely as you can to only set older definitons to the affected SEP 14 clients.

If in doubt, set all SEP 14 definitions to that 2 August r2 selection!

 

I can confirm going back to 20170802.002 solves the problem and the Win2K8 servers remain stable. 

This is the second time within 3 months this happens for us:
https://www.symantec.com/connect/forums/security-update-137-causing-bsod

How will you prevent this from happening again?

/ Torsten

 

Just posting a quick update: definitions will be posted later on today which roll back the SDS 1.4.1 engine.  Once those are available, all that will be necessary will be to run LiveUpdate/get those new definitions to the computers.  

Another day struggling with Symantec Endpoint. Several customers with a Citrix environment were unable to logon.

I agree with Torsten, how will you prevent this from happening again?

Hi we are seeing the exact same issues with our user base.

So far 100+ PC effected, although we are on 12.1.6 RU6 MP5. Issue only started once we started upgrading users DLP version from 12.5.3 to 14.5.

 

Has anyone else had similiar experience?

 

Thanks

Emilios

YL Lam i see you are also on 12.1. Are you experiencing the exact same symptoms as described above?

Random total crash of the PC where even the system clock stops and no mouse or keyboard input is possible.

Not able to connect via RDp or connect remotely to C drive of effected PC, yet it does ping fine.

 

Thanks

Emilios

8/2/2017 r8 brought down my entire Citrix enviornment.  When will new defs be released that resolve this issue?  We were here all night as our company was down until we tracked it down to SEP and restarted in Safe Mode and renamed the Symantec folder so it wouldnt autostart.

I have same issue, clientes with Window 7 and updates of 2 august. Will today new signatures solve this problem???

Tried the latest Rapid Release updates and no go.  Still forzen.

Hi,

 

make a dowgrade to SEP 12.X and It works with last defs , need to wait a fast resolution by symantec

It's urgent SYmantec

regards

 

 

Hello all,

Good news - the certified definitions which correct this issue should be available within the next 60 minutes or so via LiveUpdate. The definitions which include the latest signatures and a known-good SDS is "20170803.006" - 3 August, revision 6.

New engines like this SDS receive testing before their release, but there's no way to try them accurately with every OS, configuration and product combination.  It's a good idea to try them in your environment before their widespread release, and report any issues to Symantec Technical Support.  You can do that with our Early Adopter Server. Here are the details:

Test new Endpoint Protection definitions before they are released to production
http://www.symantec.com/docs/TECH246341

. 

My SEP agent crashed after I rebooted today. Repair or boot did not fix. It was a windows 8.1 with 14 MP2. Running symhelp to clear up the definition brought it back up. Was this related or just a random coincident?

Doesn't sound like the same thing.

BTW, those new definitions are now availble via LiveUpdate!  They are replicating to all LU servers worldwide, which can take a little while.  If LU does not bring them to you, try again after a few minutes.  

 

 

 

Thanks for the update!  I think it’s unrealistic to expect users to test every def update.  Maybe engine updates like these should be left to Version/MP updates that are tested in most environments before rollout.

Do you recommend to download new 3/8/17 r6 definitions and deploy it or stay at 1/8/17 ver. definitions and see what happen in next hours?

For everyone's info - none of the steps in the technote provided worked for one of the four servers that experienced this problem.  Instead of the recommended registry change for the SRTSP service working, we needed to change the SepMasterService to 4:

 

HKEY_Local_Machine\SYSTEM\CurrentControlSet\services\SepMasterService\Start set to 4

 

This was the only thing thatt worked for one of them.  For the other three, changing the SRTSP service to 4 worked.  Peforming the back date of SDS didn't work on any of them.

We may have other servers affected by this problem, but so far we are aware of only four.  Maybe now I can start my day, after spending the first four hours working on this...

Download and use them.  Try them out first on a smaller ("test") client group, if you wish, just to make sure that they won't cause any trouble.

Just got the fun bit of news that our DMZ SEP server (Windows 2008 R2) is no longer reachable from this issue...

Got love the passive aggressive overtones. Sorry to hear everyone has to actually work today.

@Mick2009 Will running intelligent updater in safe mode also fix the problem? If so, that seems like less hassle.

Agreed this has been a bit of a problem, Symantec. v14 MP1 on Windows 7 and this SDS update almost guarantees a lock up or hang. I wonder if I should just update any MP1 to MP2 anyway?

Will updating to MP2 solve the issue completely?

No, I don't think so, Symantec's TAC told me that all 14 versions where involved.

I can confirm running the Intelligent Updater released today, while in Safe Mode with Networking, will work.  Run the updater in SMwN and reboot for any systems that are still unresponsive.

Awesome BigAnvil! Symantec should update the post to with steps on how to use intelligent updater. Would speed up the recovery process quite a bit.

Intelligent Updater:

Here is the link to the TechNote for it, which also contains two links to the downloads (one for 12.x and one for 14.x).  Once on the downloads page, make sure you click the link for the appropriate version (x86, x64, etc).

https://support.symantec.com/en_US/article.TECH102606.html

After you download the updater, it couldn't be easier.  Copy it to the local machine or a network share, if you will have access to it from the machine you will boot into Safe Mode with Networking.  Run the executable, it will ask you if you want to check for updates to the version of Symantec Endpoint Client that's installed - choose Yes - and within 15 seconds it will tell you it finished.  The installed client generally reflects the new definitions within 30 seconds after that.

Only thing left to do is reboot, and you're fixed.  The updater made fixing the problem servers easy but it still took all day for us to resolve everything - we chose to be over cautious and check each individual Server 2008 machine to be on the safe side.

Glad this particular incident is over.

Not wanting to assume anything, does anyone know if all Windows OS's versions are affected?

Also was it all Revision of the Aug 2nd update that where affected by the Bad SDS Engine update?

 

 

Hi Emilios,

I am on 14 MP2 but some computers are still using 12.1 client. There wasn't any issue with 12.1 clients.

Specifically, out of about 300 online computers, 8 were affected (all client version 14MP2 Build 2415).

@ Symantec support - please note that we are also having this hang issue on Windows server 2k8 and it is before the released of the SDS1.4.1 engine so seems that this is still not fixed in 14 MP2.

This KB article references random system hangs and that this is fixed in SEP 14 RU1, which I assume is the next release of SEP 14:

http://www.symantec.com/docs/TECH247345