Endpoint Protection

 View Only
Expand all | Collapse all

SEP 14 Bug - Renders Windows 7 Unable to Login

Migration User

Migration UserDec 21, 2016 10:21 AM

Migration User

Migration UserDec 23, 2016 04:13 PM

Migration User

Migration UserJan 11, 2017 12:56 AM

Hans Seerden

Hans SeerdenFeb 10, 2017 01:50 AM

  • 1.  SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 02, 2016 04:18 PM

    Several of our Windows 7 Professional PC's, 32-bit are unable to login to Windows after SEP was updated from 12.1.7 to the latest 14 build. Restarting in safe mode and uninstalling SEP resolved the issue. 

    I have just disabled Generic Exploit Protection enterprise wide in hope this resolves the issue. We did not see this bahavior in any of our test machines. 

    Has anyone run into similar?



  • 2.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 02, 2016 05:00 PM

    Haven't seen this specifically but there are a couple threads related to 14 issues:

    https://www.symantec.com/connect/forums/bsod-after-upgrade-sep-14

    https://www.symantec.com/connect/forums/symantec-endpoint-protection-14-and-generic-exploit-mitigation

    Either way, you'll want to get a case open so they can get logs.

    What's is different about your test machines?



  • 3.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 05, 2016 08:08 AM

    Nothing that I know of are different between the test machines. They test machines are just a sampeling of  production machines. It is very strange that the issue just started Friday. SEP 14.x was pushed out to most of out users over 2 weeks ago.

    I will open a ticket with Symantec today, thanks!



  • 4.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 05, 2016 08:36 AM

    At this point the issue is spreading and I have disabeled GEP, Fireall and IDS/IPS.



  • 5.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 05, 2016 08:39 AM

    Has that corrected it? What happens when disabling each component one by one? This seems to be more of a firewall issue. Are you getting error messages or timeouts? 



  • 6.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 05, 2016 09:48 AM

    It may have corrected the issue, testing now.

    First the PC comes up with a black screen and says "This Version of Windows is not Genunine" and I have to reboot. We have never had an issue with that previously, and after the reboot Windows is activated again. You can enter username and password in, the screen then freezes for 30s - 10 m, and progresses to welcome. I have seen it login with no network connectivity, or never actually login.

    For a tempory fix, a reboot into safe mode with networking, logging in and then rebooting appears to let users login for the next reboot.

    There have been no policy changes from 12.1.x, and clients were on the latest version. There are no errors in the Windows or Symantec logs. No error messages or timeouts.



  • 7.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 05, 2016 09:50 AM

    So was one specific component disabled or the ones you mentioned above?



  • 8.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 05, 2016 01:03 PM

    All the components listed above. Disabling GEP and delaying AutoProtect until Symantec starts did not appear to be effective.

    Disabled so far: GEP, Firewall

     

    Thanks!



  • 9.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 05, 2016 02:22 PM

    we are having this issue, but it only seemed to occur on about 4 machines.  once machine had the same issue twice.  the machine had the log in issue, we were able to log in via safe mode.  however, rebooting still experienced the issue.  We reblasted the machine and installed a fresh copy of ver 14 during the blast. it still happened again.  i would love to know what to try.

     

    any other thoughts. 



  • 10.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 05, 2016 02:23 PM

    Disable all components. Enable one at a time, testing each time until you hit the component that causes the issue.

    I'd also suggest getting a case open so root cause analysis can be done.



  • 11.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 05, 2016 06:22 PM

    We have had issue with our upgrade as well.

    Rebooting will casue the machine to hang at applying security settings or the welcome screen.

    Rebooting to safe mode and logging in locally, then running a check disk,

    then you can log in to the newtork again.

    But on reboot same problem appears.

    do it all over again but this time on reboot uninstall SEP altogether, then reboot.

    Install 14, reboot. then the problem goes away for us.

     



  • 12.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 06, 2016 10:56 AM

    I am currently having the exact same problem. PCs will refuse to login and clicking anything on the login screen causes the computer to completely hang. This includes clicking the shutdown button. This problem started on December 1st. Our school district PCs were upgraded from version 12.1.6 to 14 on November 16th. 

    Things that we know: When a PC experiences this problem it is possible to sign into the PC in Safe Mode. Rebooting out of Safe Mode reproduces this problem immediately. Going back into Safe Mode and forcefully removing SEP using the clean wipe utility allows the PC to boot properly and login again... 

    When the computer is in this locked up state it is pingable if you know it's IP but third party VNC remoting utilities (we use Novell) do not work. In SEPM you also cannot see that this PC is on.

    I contacted Symantec to create a case report and the phone tech had me create a new default group and place one of the problematic PCs in this group to see if one of our policies was causing the problem. Unfortunately despite sitting for about an hour, this PC is still in this locked up state and SEPM does not see that this PC has checked in. I will leave it as support has instructed me to do so for up to 24 hours.

    This problem is extremely frustrating. With over 2000 PCs, it has become a minefield trying to determine which PCs are broken. Every day I am getting new reports with more and more PCs in this state. Re-imaging the PC with a clean install of version 14 may or may not resolve this problem and requires more testing.



  • 13.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 06, 2016 11:00 PM

    Same situation here on around 10 PCs so far.  Started on or around December 2 on existing installs of 14.  Same steps to resolution (safe mode / cleanwipe).  Opened case 11417819.

     



  • 14.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 07, 2016 09:58 AM

    i have also submitted a case, #11420097

    Our scenario is a follows. so far only a handful of machines have been affected.  all have had issue booting. they start, get to the loading screen but never reach the ctrl+alt+del screen.

     

    it is important to know that all clients were upgraded from 12.1.6 to 14.

     

    booting in safemode does work (except for one that we could not get to boot into safe mode)

    we added a reg key to allow windows installed to start in the safe mode environment and removed the sep 14 client. the pc was then able to boot successfully.

    i am running windows 7 ent x64 on sp1 with the most recent MS updates as November's patch tuesday (to my knowledge)

    office 2010 x64 as well.  i still have GEM enabled because on of our tests before removing the sep 14 clients was to disable GEM. however, unchecking the box for that client did not allow a normal boot.  

    so far, removing and pushing a fresh copy of the sep 14 client has not seen the issue reoccur in the machines.

     

    i have been unable to determine what causing the issue.  any insight from anyone else whould be welcomed.

     

    thanks

    ian

     



  • 15.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 07, 2016 10:19 AM

    Does this occur on a fresh 14 install ir only when upgrading from 12.1?



  • 16.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 07, 2016 10:22 AM

    we have had roughly 20 out of 200 do this

    symptom: win 7 workstations with sep 14 stuck at applying settings, seems related to 12/2 definitions

    our findings so far....

    1 - seems to be related to 12/2 definitions

    2 we had some success with powering up the PC with the network cable unplugged, wait 5 mins at applyling computer settings and then you get the login prompt - login - reattach network cable and update definitions

     

    in othere cases we have reimaged

    in other cases we were able to choose last known good, then safe mode and use msconfig to walk through ever increasing boot modes, diagnostic first, then enable selective, etc. ( This process has not been consistent) finally, uninstall symantec and reinstall



  • 17.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 07, 2016 10:31 AM

    For us, it is an upgrade from 12 for the most part.  However, the upgrade was done using the managed client with the "cleanwipe" option.  We upgraded from 12 to 14 intentionally because about 30 percent of our 12 managed clients were missing the shield in the system tray.  The recommendation from Symantec was to push 14 using the cleanwipe option.  See case id: 11230371.  All of them would have been cleanwiped during the installation / upgrade process because of my packaged, managed client.



  • 18.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 07, 2016 10:35 AM

    My case# 11414424

    I am having even more computers fail today... I've begun the slow march to down grading the clients to 12.1.6 as a work around for PCs that are still responding and reimaging the ones that are not. 

    In my environment I can tell the following is happening:

    > If a PC upgraded from 12.1.6 to 14 they will eventually freeze at the login screen requiring a safe mode/clean wipe to remove the SEP client.

    > If a PC was newly imaged with a fresh install of SEP 14 the PC login screen will function but a new scenario occurs where the wired and wireless NICs go silent. Logging in locally to the PC shows that the NICs have a connection however when IPconfig is used in CMD it says that media is disconnected. Network and sharing center will show a connection but when attempting to look at more details for the IP, Subnet, Gateway, etc the window will be blank. Attempting to connect to wifi produces the same result and the PC effectively goes deaf on the network. Uninstalling SEP and rebooting brings the PC back to life.



  • 19.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 07, 2016 11:41 AM

    We are experiancing the same issues. We upgraded SEP 12.1.6 to SEP 14 and there were no policy changes in the package. Before deployment I tested on a handfull of machines with no issue but once the push was initiated we started to get some calls of users unable to login after the new client was installed at next reboot. We have over 1400 users and so far the failure rate is about 8%. So far from what I seen is either the user gets stuck on the welocome screen after login, or it will loop until the pc reboots automatically. There were different workarounds for us to get the user back online and such steps were same as mentioned in this post, but the fix is never consistent. I opened a call and Symantec is reviewing the logs I provided. But not happy having over 1400 users and facing such an issue!!



  • 20.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 07, 2016 11:42 AM

    Case: 11413620

    From my experience only upgraded clients are affected.

    One of my customers is experiencing this on about 1-2% of all Windows 7 x64 machines. Funny thing is, if you run cleanwipe and do a fresh installation everything works fine.

    And we can't find any real pattern. The most annoying thing is that it is close to impossible to reproduce. If you run cleanwipe the computer is healthy again.

     

     

     

     

     



  • 21.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 07, 2016 01:22 PM

    This does seem to be correlated to 12/2 definitions and definitions in general as 12/2 was our first report.  In addition, we're getting reports now of managed clients not having the "green dot" and LiveUpdate or "Fix" doesn't install definitions.  A reboot does seem to fix this separate glitch.   The SEPM console had an alert where 12 % of systems were not getting Proactive Threat Detection defintions but that seems to be resolved today in the console report.  Many of the problematic (i.e. bricked) computers report "A security risk remediation or new content download requires this computer to restart" or "virus defintions are missing" when in Safe Mode.  There were no policy changes from version 12.  Testing also didn't indicate any issues and many version 14's were installed well before 12/1 or 12/2.  For our installs, we do not use the firewall -- only Virus / Spyware and Proactive Threat Protection.

    sep1.JPG

    sep2.JPG

     

     



  • 22.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 07, 2016 01:23 PM

    Correlated per Symantec support or through your investigation?



  • 23.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 07, 2016 02:04 PM

    Our investigation internally and reports in this thread only.  Have not heard back from Symantec on my case.



  • 24.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 07, 2016 02:28 PM

    all of our clients that have this issue also get the "A security risk remediation or new content download requires this computer to restart​" dialog box when booting into safe mode.  interestingly enough, i did nto see the second windows.  also inveresting is that once we log into a machine that get the "A security risk remediation or new content download requires this computer to restart" dialog box, if we reboot into safe mode again; after logging in, it never makes it to the desktop but rather shows shutting down and reboots to have the same process loop. 

    all of mine are win 7 ent x64 w/sp1. no windows 10 desktops have this issue as far as i can tell.  i can not confirm if 12/2 has any significate part to play here.



  • 25.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 07, 2016 04:56 PM

    it seems only for an upgrade, not from a fresh install as far as i can tell. 



  • 26.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 07, 2016 05:51 PM

    I have spoken with Support. They are investigating the issue and they believe they have a workaround for the logon issue. There seem to be some kind of file lock.

    Exclude as described in this link

    https://support.symantec.com/en_US/article.TECH236543.html

    If you have access to a machine with the problem it is crucial that you get a memory dump so Symantec gets more info. My memorydump was not complete :/

    In safe mode activate full memory dump using the procedure below:

    https://support.symantec.com/en_US/article.howto31321.html

    You can force a bluescreen and memory dump at the time of freeze by enabling this registry key and then holding down the rightmost CTRL key, and press the SCROLL LOCK key twice.

    https://msdn.microsoft.com/en-us/library/ff545499.aspx

    Upload the memory dump to your case.

    Torb

     



  • 27.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 08, 2016 10:34 AM

    Thank you Torb!  You got us the critical info before Symantec did! 

    We have pushed the exclusion as noted and will keep watching but this is promising.  We have had about 3 or 4 seemingly random workstation 'lockups' that may be related (as noted in the bulletin) - A note that a few of our workstations have had several recurrances - we have reinstalled SEP 12 on these few machines and will wait till Symantec fully resolves this issue before pushing SEP 14 again.

    We will be watching the technotes for a final resolution.

     



  • 28.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 09, 2016 02:32 AM

    After applying the exclusions per the article is the issue fixed now? or is it still appearing?  Thanks



  • 29.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 09, 2016 08:28 AM

    Too early to tell as we have a few thousand endpoints, policy was set yesterday and it seems like a restart is needed to determine if the computer experiences the issue.  If we get tickets today, I will update.  However, any computer that experienced this issue previously was safe-moded and clean-wiped and put back to 12.



  • 30.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 09, 2016 08:36 AM

    Thanks for the reply Robert, please keep this thread updated as after applying the exception if it made any difference. Please let us know about today progress. Thanks



  • 31.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 09, 2016 08:56 AM

    We also set the policy yesterday early during peak power on/login times.  Yesterday we only had 2 incidents (our norm had been over 10 in a morning).  This morning we do have one so far (not a good sign) and I can confirm that machine is powered off every night and was indeed powered on and used the past few mornings.

    Will update as we have more information.



  • 32.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 09, 2016 08:57 AM

    i want to also state that the exclutions did not work on our end as well. now update from our side.  a machine that has been having the same issue, had the av client (thats was upgraded from 12.1.6 to 14) removed manually in safe mode.  the 14 client was then pushed to the machine as a fresh install.  it was rebooted yesterday around 2:30pm est and sat at the loading screen spinning.  this was left alone overnight and still was sitting on the loading screen spinning.

    we are going to get a memeory dump here today.

     

    should i post the memory dump here?

     



  • 33.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 09, 2016 09:08 AM

    Add it to your Symantec case. Just remember to take the full memory dump



  • 34.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 09, 2016 09:22 AM

    @Ian

    When you reinstalled SEP 14, did you do so with a new installation package that contained the exclusions?

    If you used your old package that didn't have the new exclusions it might still crash if it rebooted before it got a chance to get the new policy from the SEPM.



  • 35.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 09, 2016 10:38 AM

    We have normally between 5-7 workstations a day do this out of over 250 workstations.  After pushing the exclusions on Wed night, Thursday was quiet, today however, we've had 5 so far.  No rhyme or reason for why they do this, so we have backed down from 14 to 12 in a few branches.  If this isnt figured out soon, we may just bite the bullet and back off all the workstations. Anyone have an easy method to uninstall 14 in bulk?



  • 36.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 09, 2016 10:49 AM
    That's to bad. Just verify that the 5 client actually received the new policy. From our testing we found that only clients that didn't get the exclusion was crashing. Meaning that they had been offline or offsite after the exclusion was set.


  • 37.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 09, 2016 11:22 AM

    I 'believe' that all of our occurences today may have been clients that had been out of the office and had not received the updated policy.  I guess Monday will be the real test for us. 

    FYI - we copied the cleanwipe utility to everyone's local hard drive on logout the other night and that was a real time saver.  The recovery process now takes only a few minutes - safe boot / cleanwipe / install SEP12

    we went back to SEP12 on the impacted machines because we were having so many re-ocurrences - once we are confident with SEP14 we will let SEPM push it again.

    We also had one report of a user stuck at 'applying computer settings' who did not report the issue and he said that it finally went to the login screen after 4 hours

    Symantec support has been disappointing at best on this one.

     

     

     



  • 38.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 09, 2016 11:44 AM

    that i did not do.  but i did figure that a machine will get the policy before a reboot woould occur, would that not be the case?

     



  • 39.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Broadcom Employee
    Posted Dec 09, 2016 01:36 PM

    Hello Doug,

    Do you have a case opened with Symantec?  If so, please provide the case number. We have only received data from one customer so far.  To confirm it is the same issue we will need Full Memory Dump during the hang. 

    The workaround should work, however if it is not we will need the Memory Dump to make sure you are having the same hang we are currently troubleshooting.

    Sincerely,

    John Owens



  • 40.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Broadcom Employee
    Posted Dec 09, 2016 01:40 PM

    Hello Everyone,

    If the workaround is not working for you please verify the clients have received the policy.  If they have received the policy and continue to hang at login we will need a Full Memory Dump to determine next steps.  Please open cases with Symantec as well to better track this issue.  We appreciate your patience while we work to resolve this issue.

    Sincerely,

    John Owens



  • 41.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 09, 2016 02:15 PM

    Hi John, as of now how many customers have reported this issue? and the only workaround for all of the customer is it to add the mentioned exception? Thanks



  • 42.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Broadcom Employee
    Posted Dec 09, 2016 02:27 PM

    Hi there,

    Under 20.  Yes, currently the only workaround for this is to add the exception.

    Thanks,

    John



  • 43.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 09, 2016 03:14 PM

    I checked the 5 that had issues this morning, and all of them have the exception in the registry.  So in our case, the exception did not work for us.  We will see what tomorrow and Monday brings, but in the mean time, we will go back to 12 on affected workstations.



  • 44.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Broadcom Employee
    Posted Dec 09, 2016 03:26 PM

    Hi Rick,

    Can you provide Symantec with a Full Memory Dump during the hang so we can check to see if you are experiencing a different issue than has been reported to us? Do you have a case open with Support?

    It would be a great help.

    Let me know.

    Sincerely,

    John Owens



  • 45.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 09, 2016 03:31 PM

    Just to be 100% safe :) So you logged into the machine while they were in safe mode without network? They had at no point since the freeze booted up and contacted the SEPM? 



  • 46.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 09, 2016 03:50 PM

    John contacted me, and I give him further information.  Our workaround for these workstations, are to pull the cat6 cable off the back after powering the affected system down, and then turn it on without the network cable, and it will allow us to come to a login screen, then we plug it back in and log in normally.  Which may point to the NIC part of the kernel? 

    The problem we have with safe mode is, administering it from afar without an iDrac port.  These workstations were used a full 24 hours after the exception was pushed before this issue arrose today again.



  • 47.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 09, 2016 04:11 PM

    Ah.. Ok. Please let us know how things work out. Your case might be something else with kind of the same symptoms. We´ve not experienced anything NIC related. It either worked or it didn´t. When we reinstalled SEP  14 it was impossible to reproduce

     



  • 48.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 09, 2016 04:50 PM

    my case # is 11415659 - we have provided a memory dump to the ticket

    I believe the workaround is working.  I think we are only dealing now with machines that have been out of the office, etc.  I confirmed that all of today's issues were machines that had not gotten the exception policy. 



  • 49.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 12, 2016 04:39 AM

    hello guys, do we have any update on this? After applying the exclusion did it make any difference or the issue is still the same?  

    Appreciate your comments. Thanks 



  • 50.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 12, 2016 08:11 AM

    it works in some cases, it doesnt in others. test for yourself



  • 51.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Broadcom Employee
    Posted Dec 12, 2016 10:47 AM

    Hello,

    There is a potential that two seperate issues are happening with the same symptoms.  If the workarond is not working in some instances, please open a case with Symantec and provide a Full Memory Dump collected during the hung state.

    Thanks,
    John Owens



  • 52.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 12, 2016 11:41 AM

    i have pulled a mem dump and a sysdiag from a machine. the sys diag was taken in safe mode and the mem dump was taked during the loading screen (just before it would normally get to the ctrl + alt + del screen.

     

    John, i sent you a PM with these files as they were too big to email and the file upload website is blocked by our enterprise.  if the files in the PM dont work, i will try something else.

     

    again, case# 11420097



  • 53.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Broadcom Employee
    Posted Dec 12, 2016 02:39 PM

    Hello Everyone,

    For cases where the exclusion is not working, could you please try the following exclusion as a workaround?

    \Windows\rescache\rc0045\Segment2.cmf

    If that fails to resolve the issue please try:

    \Windows\rescache\

    Please continue to open cases on these hangs.

    Sincerely,

    John Owens



  • 54.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 12, 2016 03:30 PM

    FYI - the "C:\ProgramData\Symantec\Symantec Endpoint Protection\14.0.1904.0000.105\Data\Definitions" exclusion has apparently worked for us.  We had no incidents thsi morning. 

     



  • 55.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 12, 2016 03:52 PM

    Thats good to hear Doug. so it worked on almost all of the machiens which were having problem right?

     

    Hi John, Can you please provide the full path for the exclusion you mentioned? Thanks 



  • 56.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 12, 2016 06:02 PM

    If any of you are stille experiencing hang after setting the exclusion, boot the computer into safe mode without network and check the registry key: HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Symantec\Symantec Endpoint Protection\SMC\SYLINK\SyLink\Serial Number

    The date in the serial number should be from when you set the exclusion or newer.

    The computers we have found that are still freezing are using a policy set from before the exclusion was set due to being offline or offsite.

     

     



  • 57.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 13, 2016 09:30 AM

    Unfortunately, this suggestion did not apply to us.  The date in that registry key is of the day we made the exclusions, yet we're still having the issue daily.



  • 58.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 13, 2016 11:23 AM

    Hi Ian, do you have issue on all of the machines which were upgraded to SEP 14 or you are having it on some specific machines?  Thanks



  • 59.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 13, 2016 03:03 PM

    Update:

    We noticed far fewer reports of problematic computers but two more have come in since I added the first exclusion.  One was fixed manually before I could observe it.  However, in the SEPM console, it did appear to have the initally-updated policies that excluded the folder / sub-folders "C:\ProgramData\Symantec\Symantec Endpoint Protection\14.0.1904.0000.105\Data\Definitions" but I could not verify on the machine.

    Another came in today and it also had the initally-updated policies with the first-suggested folder exclusion.  The user could still not log in even after a reboot or two.

    Since this was a remote user and not a lot of time to test, I excluded \Windows\rescache\ as folder exclude / sub-folder exclude.  After a reboot, I observed in SEPM that the client got updated policies.  The user was then able to log in.  

     

     

     

     



  • 60.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 13, 2016 03:33 PM

     

    we did not have issue will all machine, but we are about 1-10 now that do have the issue.

    we applied the \windows\rascache\ exception to one test machine.  it still got the issue after the exception

    this machine had had the 14 client fresh installed a few times now.  we also verified that the test policy was applied to the machine in safe mode

    a few other machines that we needed to remove and reinstall the 14 client have not had the issue occur again.

     



  • 61.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 14, 2016 04:16 PM

    my one computer (that i applied the windows\rascache exclusion) is still having issues after a reboot.  it does not matter that the client was a fresh install. 

     

    we have since decided to roll back to 12.1.6 because it is too random.  the rollback to 12.1 will happen tonight.

    we will keep a machine or 2 around for testing but otherwise 99% will go back to 12.1



  • 62.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 14, 2016 04:21 PM

    we had around 3 issues this morning.  They were repeat machines.  I just added the second (rescache) exclusion.



  • 63.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 15, 2016 01:11 AM

    Hi Doug, after adding the second exclusion for rescache are you still having the problem?



  • 64.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 15, 2016 04:11 AM

    We have the same problem overhere: upgraded 125 windows 7 x64 SP1 machines with packages from the SEP Manager and about 35 machines did not restart.

    As far as I can see the problem does not occur when I install SEP 14 as a stand-alone .exe file and reboot the machine immediatly after upgrading. I do notice that I have to reboot a machine twice ??? That is strange.

    After pressing Ctrl-Alt-Del the system hangs. I am very busy now with resolving the issieu of all these PC's.

    I will try to make the exclusion, see if it helps. I have also opened a case with Symantec.



  • 65.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 20, 2016 08:31 AM

    Hello guys, what is the update on this? Are you still facing the same issue of hang or is it fixed? Appreciate your responses. Thanks 



  • 66.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 20, 2016 08:47 AM

    We rolled our PCs back to 12.1, as we were having to fix 1-3 machines daily.  We've kept one computer that previously had problems on the new version and it's been running without issue.  However, that's not a great test because the problems were seemingly random across our environment (100 or so computers, plus servers).  We're waiting on a fix to redeploy the upgrade across the rest of our PCs and servers.



  • 67.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 20, 2016 08:51 AM

    Unfortunately, I ended up giving up on support. Not one tech asked me for logs and when I referenced the tech article or this forum post I was ignored; given scripted responses to do first level troubleshooting. I had to either re-image my PCs or do an uninstall of version 14 and then a downgrade to version 12. Based on testing version 12 does not experience this problem at all. I would also note that based on extensive testing this problem is not just isolated to PCs with upgraded installs from 12 to 14. We have seen clean installs randomly break the network stack on computers that were freshly imaged and then installed with version 14. I mentioned this in a post above. At this point a majority of PCs on my network are downgraded so I am no assistance to support even if I wanted to help further. 



  • 68.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 20, 2016 08:52 AM

    Is Symantec support providing any detail to this and what exactly is causing it? Is there a fix in the works and I can only assume it will be a new version to correct this? Is it specific to OS, hardware, drivers, etc? Is it a specific SEP component causing the issue?

    As of now for me, 14 stays in non-prod.



  • 69.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 20, 2016 09:04 AM

    Actually the worst part is there is no clear explanation from support regarding what is the reason for this behavior and what is the permement fix for this. I belive this issue is now know for 3-4 weeks and by now we should have some clear explanation from Support as what is the permemenent fix for this.

     

     



  • 70.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 20, 2016 10:18 AM

    Support was definitely less than responsive on my case.  It was probably 48 hours from opening it that I got my first response.  They never asked for logs / anything.  I found the link to the exlusions on this forum before Symantec Support replied with a link to the same tech article - probably 24 hours later.  Here was one reply from support that was a bit disheartening:

    Hi Bob ,

     Thanks for your response , heartening to  know that exclusions are fixing the problems.
     The design team might fix  it permanently in  our next major release . Please let us know if we can close ticket  &  stop following the case or  do you still want it under observation.

     Thanks &  Regards
     Rajeev

    I indicated that it was way too early to close the ticket and that I wasn't sure if everything was fixed.  It might be fixed in the next major release????!! 

    To answer the question about if the exclusions seem to be fixing the problem, the exclusions (two) do seem to be addressing the problem but I don't consider that as fixed -- I consider that as temporarly worked around until a fix is released.  I had two machines yesterday where after a reboot and them being online and getting policy, I was able to log in.  We've not had to Safe Mode fix for a while.  

     



  • 71.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 20, 2016 10:19 AM
    So, after forwarding a memory dump of our issue to SEP support, we excluded the following file that was hanging up the startup in our environment. C:\Windows\System32\FWPUCLNT.DLL We are going on day 5 of no issues with start up on any workstation. Maybe this can help others. Thanks to John Owens at Symantec support for helping us with our issue.


  • 72.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Broadcom Employee
    Posted Dec 20, 2016 10:54 AM

    Good Morning Everyone,

     

    Symantec released a definition update on Decemeber 13th (12/13/2016 Rev 23) that resolves one of the two hangs we know about currently.  This is where Auto-Protect was hung up on a file in  \Windows\rescache.  If you have definitions running 12/13/2016 Rev 23 or newer please remove the \Windows\rescache exception.

    The second hang is still being worked on.  This hang is in the C:\ProgramData\Symantec\Symantec Endpoint Protection\14.0.1904.0000.105\Data\Definitions folder. Please keep this exception in place.  We currently believe this will be fixed in 14 MP1.  Once this is confirmed I will update the forum.  Current soft ETA for this is February.

    Sincerely,

    John Owens

     

     



  • 73.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 21, 2016 02:14 AM

    I also opened a case. I was told to wait till the next release or roll back the machines with version 12.

    It looks like the problem only exists when I install v14 by the SEPM, when I use an executable the problem does not occur. So, I embedded the executable in a Altiris job to be rolled out on clients and then it works fine.

    I also added the 2 exclusions to the exeptions.

    I am not happy with the response of Symantec, I think poor testing gave us this problem.

    Let's hope the new releae will be out very soon.

     



  • 74.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 21, 2016 03:50 AM

    Hello John,

    Our defenition updates are currently version 20-12-16 r20.

    Is it sure to remove the exeption on \Windows\rescache?

    Maybe just to be sure not to get all the problems back again to leave it there until MP1 is out and distributed?

    Kind regards,

    Hans Seerden



  • 75.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Broadcom Employee
    Posted Dec 21, 2016 10:13 AM

    Hello Hans,

    Yes.  For that Hang the definitions you have will resolve the issue and you can remove the windows\Rescache exception.

    It is okay to leave it until SEP 14 MP1 comes out as well.

    It is really up to you and what makes you feel most comfortable.

    Thanks,

    JOhn Owens



  • 76.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 21, 2016 10:21 AM

    John Owens,

    When is MP1 coming out?



  • 77.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 21, 2016 12:09 PM

    We have upgraded 2000 endpoints to SEP 14. After we added the exclusions we haven't had any problems. From 30 a day to zero :)

     

     



  • 78.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 21, 2016 02:57 PM

    Thats a good news TORB. Just one thing, have you also enabled Generic exploit mitigation on these 2k endpoints? has there been any false positives you have observed with it? Thanks



  • 79.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 21, 2016 03:13 PM
    We use all components except the email plugin. So far zero false positives and no issues with GEM or ML. There has been a big increase in real malware detections so it seems like the protection has gotten better.


  • 80.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 23, 2016 04:13 PM

    Is there an update on resolution to this?



  • 81.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Dec 28, 2016 03:46 PM

    I think for now the only work around is to put in the mentioned exception :)



  • 82.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Jan 06, 2017 11:02 AM

    Any update when the first revision to SEP 14 is coming out

    Thanks 



  • 83.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Jan 11, 2017 12:56 AM

    MP1 will be relasing mid of Febuary. 



  • 84.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Feb 01, 2017 11:22 AM

    Has there been ETA date for the MP1 release?



  • 85.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Feb 09, 2017 11:30 AM

    Is this still set for mid February? Any concrete dates yet? I need to update my SEP server for Win10 (1607) compatibility reasons, but was waiting to see what happens with this new MP for SEP 14. Thanks.



  • 86.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Broadcom Employee
    Posted Feb 09, 2017 11:57 AM

    Hi All,

    The current ETA is February 17th.  I will update this thread if the release slips for any reason.

    Thanks,

    John



  • 87.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Feb 10, 2017 01:50 AM

    Thanks John!



  • 88.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login



  • 89.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Broadcom Employee
    Posted Feb 21, 2017 02:20 PM

    Hi All,

    Please remember to remove the exclusions you have in place to workaround the issue once SEP 14 MP1 is in place.

    Thanks again,

    John Owens



  • 90.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Feb 28, 2017 01:39 PM

    Hi,

    Just curious if you've updated your SEP server and clients to v14 MP1 and if so did you run into any issues? We're running 12.1 RU6 MP5 and getting ready to upgrade to 14 MP1.

    Thanks



  • 91.  RE: SEP 14 Bug - Renders Windows 7 Unable to Login

    Posted Mar 18, 2017 06:22 PM

    Hello Eric

    Have you got anywhere with a resolution on this one? We are experiencing the same issue in our environment when Imaging with SCCM. After the reboot when SEP updates we can no longer login as network connections are disabled. Local admin does not even work. The only thing we can do at this point is to have someone who logged already logged in prior to the reboot log back in, remove 14 and reinstall 14.