Video Screencast Help

MR3 Locks up Server 2008 file shares

Created: 29 Sep 2008 • Updated: 21 May 2010 | 200 comments

An issue supposibly fixed in MR3 is still happening to me.

 

The auto-protect will 'malfunction' and file shares will become unaccessible on server 2008.

 

I'm running Server 2008 Standard x64. Another administrator i know experienced the same issue today on x86 of server 2008.

 

My setup has DFS installed on it, and i've excluded the DFS roots as well as any replicating folders as well as tons of other folders.

 

The only thing I see in my event log is that virus definitions were updated recently...

 

Anyone have any suggestions or know of any virus product that doesn't destroy access to your file shares?

 

 

 

 

 

 

Comments 200 CommentsJump to latest comment

Chinchilla's picture

FYI. I have just been informed that MP1 will likely be released on Monday. Whether or not this will contain a fix for this issue, I don't know.

Paul Murgatroyd's picture

Chinchilla is correct, MR4 MP1 is slated to be released internally on Monday, it will generally hit the web and customers networks a few days after that.

 

The fix for AutoProtect will sadly not be in MP1.  It was simply too late in the testing cycles to get it in without significantly impacting the timescales of MP1, and we have several customers depending on us for key fixes related to their environments in MP1.

 

At this point in time, the fix itself will be officially shipped in MR4 MP2 later this year.  HOWEVER, because we recognise the serious nature of the issue, we are going to release a hotfix for those customers who have been affected by this.  As QA are still finishing up on MR4 MP1, we haven't sat down to work out timescales yet, but they will need to fully certify it before we can ship the code - initial testing within engineering has proved very positive though.

 

As next week pans out I will post more up to date information with respect to timescales, but in the meantime, the workaround posted about disabling cache re-scanning works for most.

 

On the subject of asking customers to install the product to crash their servers and send the dump files - we acknowledge that this is a massive request, and we would never normally ask this of you but we have seen so many different causes and reasons for crashes with SMB2 (not all of them Symantec related) that we wanted to make sure we got every case and dumpfile possible for our engineers to analyse.  As most hangs occur under load, it is difficult to properly replicate "normal" server loading in a lab - yes, we can run scripts, etc. which simulate it, but in reality its never quite as good as the real thing.

 

 

 

 

Paul Murgatroyd
Principal Product Manager, Symantec Endpoint Protection
Endpoint twitter feed: http://twitter.com/symc_endpoint

ShadowsPapa's picture

Hope something is done soon to allow application and device control to be used without killing WORD.......

 

I guess I'm REALY glad we don't have Server2008 here!  We don't need MORE headaches, and I'm afraid that would be just what they'd need to give the order "get SEP outta here".

I'm shocked I've not been told that already, as bad as our slowdowns have been since MR4.

SKlassen's picture

Bump for a hotfix update.  QA got a one day rest after release of MR4 MP1, time to chain them back to their desks again Paul.  ;P

Paul Murgatroyd's picture

QA are currently working on certifying the fix.  We are hoping to have this complete by the end of this week.  The fix will be in the form of a single binary that can be dropped onto an affected server and replace the existing file.

 

QA are certifying the fix on MR3, MR4 and MR4 MP1.

 

More news when I get it.

Paul Murgatroyd
Principal Product Manager, Symantec Endpoint Protection
Endpoint twitter feed: http://twitter.com/symc_endpoint

portent's picture

Thanks for the update Paul.

 

This is a major problem in our company.

The rest of the guys here, saw the opportunity to replace Symantec with another Antivirus vendor.

After some serious discussions, i think your safe this year. :smileywink:

 

Please remember to post when this fix is available.

 

Kind Regards

 

andrewk's picture

I would like to add my experience to this thread, which I've been quietly following for a few weeks.

I rolled out a new Win2k8 server, 64-bit, Exchange 2007 last month.  Everything was smooth until unmanaged SEP MR4 client was installed on Thursday 1/29.  On Tuesday 2/2 client called saying all XPSP3 clients were experiencing slow logins, and when (and if) they got logged in, server resources (mapped drives, Exchange) were inaccessible.  Client logs all revealed 1030 and 1058 errors which generally indicate a failure to process group policies, which in turn is indicative of an inability to access DFS shares.  At the server console, the OS was responsive but event logs showed nothing out of the ordinary.  Reboot fixed the problem for two days, then it cropped up again twice on Thursday 2/25.  After hours, and hours...and hours...of digging around I came across this thread and managed to get a case open with Symantec.  The suggestions were to either (a) disable Symantec services (b) uncheck re-scan cache option in auto protect (c) create exclusions for shared folders DFS shares and (d) uncheck IPv6 from the NIC.

 

Can anyone confirm if any combination (b) (c) and (d) has worked?  At present I have Symantec services disabled - and the server has been fine since.  But of course now I have an unprotected production server and Symantec is going to close the case unless I do further testing - but further testing could crash the server and make our up-until-now-very-successful-consulting-firm look pretty bad.  I'm really in a tough spot here.  Comments?  Suggestions?

 

Also - unchecking IPv6 on the NIC (which is not the proper way to disable IPv6, by the way) causes Exchange to throw all sorts of errors in the app log, although none of them appear to affect access to Exchange or it's performance and stability.  But what a nuisance when the client wants to look at application logs with you, and you have to explain these errors.

 

Help?  Comments?

SKlassen's picture

Thanks Paul.  Looking forward to it.  Hoping you can pull some strings to get it posted to the Symantec FTP (like the srtspt fix long ago), so that we don't have to go through the support hastle to get it.  :) 

Knottyropes's picture

EugB wrote:

I'm not having this problem on my W2K3 servers, so I'm not able to really contribute much...I am curious for those having this issue if disabling Digitally sign communications (if server agrees) in GP helps or not.  Particularly with W2K8 Server.

 

I've heard of a cuople of sites with XP workstations having connection problems to W2K8 servers (while Vista clients work fine) when this is left enabled (it's enabled by default).  I recall at least one poster in this thread with the same type of network, so I thought I'd toss it out there.

 

The setting is under Computer Config -> Windows Settings -> Security Settings -> Security Options -> Microsoft network client: Digitally sign communicaitons (if server agrees)

 

Just a thought.

 

Message Edited by EugB on 02-02-2009 01:41 PM

 

I have SMB enabled but only XP machine here.

Knottyropes's picture

Conestoga Rovers wrote:

Hey Knottyropes, I tried your registry tweak and 33 hours later I had to reboot my W2K3 server again.

I just went back to the workaround suggested by Symantec, disabling the File System Auto=Protect and increasing the scheduled scanning.

 

 

Try adding this reg setting and see if it fixes it. I did it on my 2003 DC that has a few shares and it works well with MR3. I also added it to my other servers as well.

 

HKLM\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters create a DWORD called TcpTimedWaitDelay=40

 

Sorry to here that. I have all my servers on MR3 (except Exchange is on SAV since it works) even the new file server that has been online since OCT. All have the reg edit in it. I have broadcom and intel NICs no teaming. no jumbo frames. no managed switches.

 

 

FYI on what is working all with MR3 installed, only AV installed.

 

2003 R2 SP2 SQL 2005 125 days (upgrade)

2003 SP2 FTP/WEB/CRM 54days (power fail)

2003 SP1 DC/SEPM/File SP3 322 days (powerfail)

2003 SP1 ISA2004 (Was rebooted this week for update)

2003 R2 SP2 File/Print/Fax/DC 20 days (ram failure)

2000 sp4 BlackberryEnterpriseServer 20 days (upgrade)

 

Paul Murgatroyd's picture

OK, true to their word QA completed certification on these fixes.

 

If you have a 2008 X64 server, please view the following KB for information and download details:

 

http://service1.symantec.com/SUPPORT/ent-security.nsf/docid/2008100113145148?Open&seg=ent

 

If you have any other platform that is exhibiting these problems then you need to CALL SUPPORT and ask them for the files in KB 2008061812370848

 

Please let me know how you get on.

 

thanks

 

 

Paul Murgatroyd
Principal Product Manager, Symantec Endpoint Protection
Endpoint twitter feed: http://twitter.com/symc_endpoint

JRV's picture

Yes, thanks! Looking forward to putting this one to bed for good.

pesos's picture

Hi Paul,

 

How can we incorporate the fix for 10.2?

 

Thanks!

Wes

SKlassen's picture

pesos:  You can't with this patch, it is for Sep 11 MR3 and MR4 versions.  If the problem exists with SAV 10.2, it is a completely separate product which would need its' own patch.  This forum is strictly for SEP/SEPM 11.X, you'll have a much better chance getting assistance with SAV in the correct forum at https://forums.symantec.com/syment/board?board.id=sav10x .

Message Edited by SKlassen on 02-15-2009 04:09 PM

pesos's picture

Paul has already replied to me here and this seems to be where the action is.  I'll create a new thread, but Paul if you could respond here or there it would be most appreciated.  The new thread is here: https://forums.symantec.com/syment/board/message?board.id=sav10x&thread.id=1621

 

thanks!

Wes

Message Edited by pesos on 02-15-2009 04:26 PM
Tdadmin's picture

What about Windows server 2003 that have the same problem ? i can NOT install symantec on my 17 DC cause it keep stop the share...

 

Thanks

 

Paul Murgatroyd's picture

Updated KB's:

Title: 'Problems accessing file shares on Windows Server 2008 64-bit running Symantec Endpoint Protection'
Document ID: 2008100113145148

http://service1.symantec.com/SUPPORT/ent-security.nsf/docid/2008100113145148?Open&seg=ent

 
Title: 'Network shares become unresponsive on Windows Server 2008 32-bit with Symantec Endpoint Protection 11 Auto-Protect enabled'
Document ID: 2008061812370848

http://service1.symantec.com/SUPPORT/ent-security.nsf/docid/2008061812370848?Open&seg=ent

 

Title: 'Network shares become unresponsive on Windows Server 2008 32-bit with Symantec AntiVirus 10.2 Auto-Protect enabled'
Document ID: 2009021611455248

http://service1.symantec.com/SUPPORT/ent-security.nsf/docid/2009021611455248?Open&seg=ent

 

pesos, are you running the latest build of 10.2?

 

For everyone who has this problem on 2003 server, you are welcome to try the binaries, but they have not been tested on that OS.  Have you reported the problem to Symantec via support channels so that we are formally aware of your issue?  We have millions of customers worldwide running SEP on Windows Server 2003 with NO issues like this.  Please make sure you are running the latest maintenance release (MR4) as there was an issue with our first (RTM) release, but this has been long since fixed.

 

 

 

Paul Murgatroyd
Principal Product Manager, Symantec Endpoint Protection
Endpoint twitter feed: http://twitter.com/symc_endpoint

ShadowsPapa's picture

Have it running on 3 DCs in Des Moines, and one in Ames. No issues, but then I followed the book and did a server install, didn't install the full client arsenal.

NO ISSUES with shares, no issues at all on the DCs.

I DID follow best practices and excluded via policy all the DHCP, DNS, and other critical DC areas.

Conestoga Rovers's picture

Paul, what about Windows 2003? Same fix applies to Windows 2003 server, too?

Paul Murgatroyd's picture

see my earlier post

Paul Murgatroyd
Principal Product Manager, Symantec Endpoint Protection
Endpoint twitter feed: http://twitter.com/symc_endpoint

bcap's picture

When I login to the licensing portal or fileconnect it only gives me the option to download 11.0.4000 (MR4).  How can I obtain 11.0.4010 (MR4 MP1) ?

 

 

pesos's picture

Hi Paul, my win2008x64 server SAV install is reporting program version 10.2.2.2 -- is this still the latest?  I downloaded it very recently.

 

Thanks!

Wes

pesos's picture

pesos SAV is not SEP

 

Umm...  I know?  Maybe that's why I referred to it as SAV and not SEP?  SAV 10.2 is the predecessor to SEP 11, and 10.2 is seeing this same lockup issue on win2008x64.

David-Z's picture

10.2.2.2000 is the latest SAVCE version for vista/w2k8.

 

Hope that helps!

David Z.

Senior Principal Technical Support Engineer, Symantec Corporation

Enterprise Security, Mobility and Management

Woodford Computers Engineer 1's picture

Hi Everyone.

 

MR 4 Locks up SBS Server 2008. I am having trouble removing Symantec Endpoint 11.0.4000.

I escalated the call Microsoft they remotely check server and saw common issue with Symantec Endpoint, I manage remover Symantec Protection Manager, but when I attempted remove endpoint the server crash.

I attempt call Symantec twice stay on telephone for 30 minutes and gave up. I am now attempting use old version clean wipe I have it did not work as it only goes up to version 10. Apparently you can get it from download CD2 but I do not think it is on there. 

 

Please can someone help Symantec problems on SBS 2008.

Paul Murgatroyd's picture

Do you still have a working install of SEP on the server?

 

If so, have you patched the files according to this KB (which is what this whole thread is about)?

 

Title: 'Problems accessing file shares on Windows Server 2008 64-bit running Symantec Endpoint Protection'
Document ID: 2008100113145148

http://service1.symantec.com/SUPPORT/ent-security.nsf/docid/2008100113145148?Open&seg=ent

Paul Murgatroyd
Principal Product Manager, Symantec Endpoint Protection
Endpoint twitter feed: http://twitter.com/symc_endpoint

Paul Murgatroyd's picture

pesos, yes, thats the latest build.

 

We haven't yet QA'd the fix on SAV on W2K8 at this point in time, but I've had a chat with the engineer who writes the AP code and he says that you should be OK to apply the files from the SEP KB's.  SEP and SAV still share the same AP code, so there shouldn't be a problem.

 

As far as everything else goes, has anyone applied the fix yet, are your servers staying up?

 

 

 

 

 

Paul Murgatroyd
Principal Product Manager, Symantec Endpoint Protection
Endpoint twitter feed: http://twitter.com/symc_endpoint

pesos's picture

Thanks Paul, I will see about applying the fix and re-enable autoprotect and report back.

 

-Wes

Paul Murgatroyd's picture

anybody any news?

 

have you patched your servers?

 

are things looking better?

 

initial reports have been very positive, I'd like to get the community view!

 

thx

Paul Murgatroyd
Principal Product Manager, Symantec Endpoint Protection
Endpoint twitter feed: http://twitter.com/symc_endpoint

Conestoga Rovers's picture

I upgraded my SEP Manager to MR4 MP1 (using the new released installation package) and enabled File System Auto-Protect on two of my Windows 2003 servers. One is still running since yesterday morning, the other one was changed today.

I want to see it first without the quick fix.

 

Paul, did you get any advice on the Windows 2003 server?

JRV's picture

Speaking for myself, I'd love to be trying this right now but am up to my a** in alligators (non-SEP alligators for a change) and need to make some headway elsewhere before I come back to this.

 

Meantime, am watching this thread for positive reports because I'd like not to be the first to dismantle the workarounds and discover that our situation isn't the one that got fixed! But I have a fairly low-risk machine to try it on and see what we learn. If MR4 MP1 is posted to FileConnect any time soon, I'll probably combine the projects.

The Zone's picture

Tried it on a 2008 64-bit server and so far so good.

 

Paul, is there any news on a hotfix for 10.2.2 on Windows 2008 64-bit? I have several customers in that boat who were exhibiting the same file share problem.

 

I understand that the 11 endpoint hotfix *should* work on 10.2.2 but is not yet certified by Symantec. I'd like it to be certified before I make any move on that front.

 

 

Paul Murgatroyd's picture

I have asked about the 10.2 side of things, awaiting a response

Paul Murgatroyd
Principal Product Manager, Symantec Endpoint Protection
Endpoint twitter feed: http://twitter.com/symc_endpoint

wroot's picture

Hi,

 

I've been following this thread for a while. So, now i have a question about upgrading process. We have MR2 MP1 Manager on WS2003 32bit and clients are mixed versions of 11.0 (some MR2 MP1 - 11.0.2010.25, others older, all 32bit). Both this server with Manager and other WS2003 32bit has 11.0.2010.25 clients. Now we are planning to launch 4 virtual WS2008 64bit servers and 1 virtual WS2003 32bit server. Can i upgrade my Manager to MR4 MP1 at once? Or should i first install MR4 on top of MR2 MP1 and then upgrade to MR4 MP1? Hotfixes are supposed only for clients? How should i upgrade the clients? After upgrading my Manager to MR4 MP1 can i just generate the new install package and push it to all clients (11.0.2010.25 and older)? And then i should apply the hotfixes to the clients on my WS2008 servers?

The Zone's picture

I assume you can just upgrade from 11.02 to 11.04 MP1 (I did 11.04 to 11.04MP1).

 

Here's the steps I followed. Do this on the Console server:

 

1.  On the Endpoint Manager stop the Symantec Endpoint Protection Management service - leave the embedded database running.

2.  Run setup.exe from the \sepm subdirectory of MR4 MP1.

3.  Once the manager is upgraded assign the 11.0.4010 installation package to your groups - the clients will upgrade transparently following steps below.

 

With respect to assigning the client package to your groups please do the following. 

1.  Log into the Endpoint Management Server.

2.  Click on the Clients button.

3.  Highlight a group.

4.  Click on the Install Packages tab. Right click Add if you don't see anything. And then,

5.  Select 11.0.4010 from the drop-down list of packages. Repeat for each group, selecting the correct version (32 or 64-bit if you've created multiple packages)

 

If you uncheck "Maintain Existing Client Features" you can dynamically control the components used by the workstations. Check the Upgrade Schedule button, select a time and duration for your updates. This will update your clients automatically.

 

Once all that is done, then do the hotfix for the file share issue which can be found here (only needs to be done on the 2008 servers)

 

32 bit...http://service1.symantec.com/SUPPORT/ent-security.nsf/docid/2008061812370848?Open&seg=ent 

 

64 bit...http://service1.symantec.com/SUPPORT/ent-security.nsf/docid/2008100113145148?Open&seg=ent

 

Hope that helps...

Message Edited by The Zone on 02-20-2009 08:49 AM
JRV's picture

Patch installed here, but am waiting for Monday to turn off the Rescan Cache workaround to avoid having to come in over the weekend if there are problems. So no results to report from here.

 

If this is of any use to anyone, following is a simple script which will install the hotfix. Saves time and ensures everything gets done, and done the same way. It worked for me, and should work for you, but if it doesn't, just remember what you paid for it, and use at your own risk! It does very little prerequisite checking. If someone makes enhancements, I won't be insulted if you post a better one. In fact, I'll most likely use your improved version as I install SEP on more WS2008 x64 machines!

 

Hope it's useful to someone.

 

Notes

  1. The script does what Symantec's instructions say to do, except it also drops a text file in the ...\SRTSP folder so the script knows if it's already been run, as will anyone who pokes around in the folder later.
  2. If UAC is enabled, you need to use Run As Administrator to run it.  (I created a shortcut to the script and set the Run As Administrator property in the shortcut to make it easier.)
  3. You need to edit the SOURCEDIR environment variable at the top of the script to point to the location of the hotfix files.

 

@ECHO OFFCLSSETLOCALREM **Instructions**REM Set SOURCEDIR to the path of the hotfix filesREM No trailing backslash, no quotesSET SOURCEDIR=\\MyServer\MyShare\MyFolderREM PreflightIF NOT EXIST "%SOURCEDIR%\SavRT32.dll" GOTO ERR_SOURCEDIRIF NOT EXIST "%ProgramFiles(x86)%" GOTO ERR_ARCHIF EXIST "%CommonProgramFiles(x86)%\Symantec Shared\SRTSP\2008_64-BIT_FIX.TXT" GOTO ERR_ALREADYRUNNET STOP "Symantec Endpoint Protection"REM Back up the old files in Common FilesCD /D "%CommonProgramFiles(x86)%\Symantec Shared\SRTSP"FOR %%A IN (SavRT32.dll Srtsp32.dll) DO REN %%A %%A.bakREM and replace themCOPY "%SOURCEDIR%\*.dll"REM Same for DriversCD /D "%SystemRoot%\System32\Drivers"FOR %%A IN (srtsp64.sys srtspl64.sys srtspx64.sys) DO REN %%A %%A.bakCOPY "%SOURCEDIR%\*.sys"REM Set a flag so we know that we ran already, in case we run it again by mistakeECHO.ECHO SEP 11 64-bit hotfix 1/30/2009 has been applied. > "%ProgramFiles(x86)%\Common Files\Symantec Shared\SRTSP\2008_64-BIT_FIX.TXT"NET START "Symantec Endpoint Protection"IF NOT %ERRORLEVEL%==0 GOTO ERR_SERVICEECHO Hotfix has been applied. Restart the computer.GOTO END:ERR_SOURCEDIRECHO Edit the script to set the value of SOURCEDIR to the path to the hotfix files.GOTO END:ERR_ARCHECHO.ECHO This hotfix is for x64 machines only.GOTO END:ERR_SERVICEECHO.ECHO Abnormal Termination: Symantec Endpoint Protection service encountered aECHO problem when starting. Check Event Log for more information.GOTO END:ERR_ALREADYRUNECHO.ECHO This hotfix has already been applied.GOTO END:ENDECHO.PAUSEENDLOCAL

 

Woodford Computers Engineer 1's picture

Thank you Paul, for your quick response. I am sure that your fix will work. I had to get Microsoft to help me to disable SRT driver for temporary fix. I have now downloaded new sep 11 MR 4 MP1. I will try next week to see if new version and fix resolve issue.

 

Regards,

 

Asif Patel

 

Woodford Computers

wroot's picture

So, i have upgraded my Management server straight to MR4 MP1 (from MR2 MP1). I didnt have to stop any service and it went smoothly. As about clients. I have never managed to upgrade them via Installed Packages. Maybe i just cant wait enough. This GUI is lacking a feedback. Deploying via Deployment server is more admin friendly :) So maybe i will try upgrading my clients this way, though i wish it could be indeed more transparent for the clients (it shows MSI install window and user has to close all applications).

Chinchilla's picture

A couple of questions:

 

1. When will the patch be released?

2. How easy will it be to distribute?

3. I notice the knowledge base articles mentions that the servers in question function normally, only the fileshares lock up, but what we are seeing here is, the whole server locks up. Are they the same issue, or is there something else that needs to be worked on?

JRV's picture

That would be 3 questions<g>.

 

1. The patch is released. You can download it from the KB article.

 

2. It's fairly easy to install manually, but I posted a batch file that may make it easier...look a page or two before this one in the thread.

 

3. The symptoms the patch addresses is that shares, and only shares, become unavailable. In my experience, RDP, ping, Telnet, IIS, etc., all continue to work while SMB (file & print sharing, SYSVOL, DFS, etc.) is broken. I can also log on from the console while it is in this state, and have SMB access to remote servers. So it's a very specific problem: If your entire server is hanging, the patch won't fix it, unless you're running some software on the server that is exceptionally sensitive to unavailable shares and capable of bringing Windows down completely.

Chinchilla's picture

1. Sorry, I should have explained better. That is not a real patch, it's a bunch of DLL's we need to copy and paste manually. For me this has to be done on hundreds of machines, with no guarantee this will work. If we had an exe or something we can deploy that did the fix, we'd be risking a lot less man-hours.

 

2. I'll take a look, it might be what we need.

 

3. These are mostly domain controllers, but I don't manage them directly. The latest issue has caused the NIC on one of them to be unreachable. If anyone has some added experience with this issue and DC's, I'd appreciate it.

JRV's picture

FWIW, 100% of my experience with this problem is on DCs. We have no other WS2008 x64 machines.

 

It's never completely hung one, it's only disabled SMB. Meanwhile, all non-SMB inbound connectivity, and all outbound connectivity, continues normally during the problem.

 

Can't speak for everyone, but looking back through this thread, and the KB article, that seems to be the commonality.

 

As for the "bunch of DLL's", I was a little surprised by the casualness of the approach, but very, very grateful for the "bunch of DLL's", as creating and testing a delivery mechanism would have delayed release of the patch. I think Symantec went out of their way to help get this out ASAP, doing some things they wouldn't ordinarily do.

 

The script I posted could easily be modified to be non-interactive, and used as a Startup Script in a GPO. I don't have enough machines to bother testing and implementing it that way, but sounds like you do. If you have a mixed environment, use a WMI Filter to scope it only to WS2008 x64 machines. Or modify the script to make that check. Make sure to remove the Startup Script after it's been run so it doesn't downgrade SEP after the fix is incorporated in a new MR/MP. And/or enhance it to version-check the files before it replaces them. And if you enhance the script at all...post the new version here!

Chinchilla's picture

Oh I'm happy they were willing to release the DLL's when and how they did. I was simply explaining what I meant, as when i was asking for a release date I was reffering to the deployable patch.

 

Sadly, it is entirely likely that this is not a resolution for my issues, which is quite bad for me.

 

As for your script, I'm considering it, but have some questions. If we do deploy it, we'd likely do it through SCCM. Or some sort of remote batch file tool like PSExec. However, I am curious because you mention non-interactive. I'm not much of script writer, but I don't see where that batch file requires any form of interactivity.

JRV's picture

There's a Pause statement at the end so the window doesn't close before the user can see the output. I think that's all. You'd not want that in a Startup script--though you might want to redirect output to a log file.

 

Conestoga Rovers's picture

MR4 MP1 on Windows 2003 server was not enough. After 5 days I had to reboot my server. I am going to apply the quick fix.

 

Better4Now's picture

We applied the DLL patch this last Sunday to address Windows XP (Vista ok) workstations locking up while working in a Windows 2008 64 bit DFS file server. 2 traces to Microsoft pointed us to Symantec issues with SMB 1.0.

 

So far so good.

 

JT

 

 

JRV's picture

Paul, I patched my MR4 servers, and yesterday, turned on Rescan Cache to disable the workaround. Hasn't been very long, but so far so good.

 

I've now installed SEP MR4 MP1 on one of them. It appears as though the MR4 MP1 installer does not downgrade the hotfix files. *.DLL's remain at v10.2.9.5, *.SYS's at v10.2.9.3.

 

Therefore, is it correct that, once the hotfix is applied, it will be permanent until a future Version/MR/MP installs a newer version of one or more of the files (which later versions will presumably also include the fix)?

The Zone's picture

Paul,

 

Any word on 10.2 and the hotfix?


Thanks-

jrudbecka's picture

We applyed Mr4 mp1 and the hotfix last friday, no problem yet.

 

This is on a 2008 32bit server, which have had the problem before.

Better4Now's picture

Update....

 

Day 5...so far so good. The sharks are dispersing....not ready to go back swimming..yet...

 

JT

Wouter Leeuwerck's picture

We had the same issue once a week with SEP 11.0.3

Applied the patch today, hope it works! 

The Zone's picture

Paul,

 

Do you have an update regarding the hotfix for 10.2.2 customers on Windows 2008?

I have several customers who are still experiencing the lockup issue. Does the hotfix released for endpoint work on 10.2.2? Is there a separate hotfix being worked on? Any ETA?

boe's picture

I'm running MR4 and it still happens!   When will Symantec release a fix or a work around at least?

petr.kasan@autocont.cz's picture
Paul,

 
Do you have an update regarding the hotfix for 10.2.2 customers on Windows 2008?

I have several customers who are still experiencing the lockup issue. Does the hotfix released for endpoint work on 10.2.2? Is there a separate hotfix being worked on? Any ETA?

I confirm this. One of our customers also still experiencing the issue, even with MR2. Today I assisted with resolving lockup on one of his fileservers.
One more info: there is no problem on Vista cilents. While XP users cannot access share, same share is accessible from Vista clients without any problem. I guess only SMB 1.0 protocol is involved in.

hsshelp's picture

Reinforcement to other users' posts...

-=-=-= sitrep/observations
- same behavior...unpredictable loss of access to both file and printer shares from our WinXP SP2/3 clients (no WinVista clients in our environment)
- noted that when the server box croaks, both the Server (LanmanServer) and File Server Resource Manager (SrmSvc) services are in the stopped state
- manually restarting above services restores file and printing services without requiring a reboot
- log files appear to yield no pertinent or correlative events.

-=-=-= server details
- Dell PowerEdge 2900
- Broadcom BCM5708C NIC with Dell-supplied v4.4.15.0 driver
- WinServer 2008 Standard 64-bit
- No DFS
- Single visible file share on NTFS partition
- Single non-system hidden file share (share$) on NTFS partition
- Access-based enumeration disabled
- 50+ shared network printers
- 200 users
- NAVCE 10.2.2.2000 (MR2)
- LiveUpdate v3.3.0.73

-=-=-= actions taken
- begrudgingly disabled auto-protect and setup nightly scan

Apologies in advance if posting specifically about SAVCE in this seemingly SEP forum is not completely appropriate, but Symantec's rep, Paul, has has been responsive here and also responded that "SEP and SAV still share the same AP code".

Paul Murgatroyd's picture

All,

At this moment in time, we are finding it difficult to allocate QA time to test this fix on SAV 10.2, due to our impending Small Business product.  As I have stated before though, both products share the same AP code and there is no reason why the fix shouldn't work... we certainly haven't had anybody call us and say it hasn't worked for them.  One thing that is causing us a problem with timings here is that we simply don't know how many people are wanting the fix certified for 10.2, so far I see maybe 2 or 3 requests on these forums, but we haven't had many official requests via support either.  If you haven't requested a 10.2 fix via your normal support channels, then please do.. it helps us prioritise our customer requirements.

 

Paul Murgatroyd
Principal Product Manager, Symantec Endpoint Protection
Endpoint twitter feed: http://twitter.com/symc_endpoint

The Zone's picture

Hi Paul,

FYI that I have opened several cases on behalf of the various customers running 10.2.2 on W2008 who are experiencing the same issue as described above.

The best I've gotten from support is that they are unsure if the SEP hotfix would work on SAV and nothing is officially supported. The other thing I got from another tech is that "we are working on testing this but have to no ETA at this time".

In the meantime, I have several customers who don't want to upgrade to SEP and just want this fixed on SAV asap. This is now going on a month with workarounds that is trying their patience.

They also don't want to do beta testing for Symantec on their production servers for a non-certified fix. They can't deal with the potential downtime at this point.

Anything you can do on your side to move this along would be greatly appreciated.

Rob

hsshelp's picture

Update to former post (just over 3 days ago)...

 

-=-=-= actions taken

- per advice from Symantec rep, Paul, applied purportably compatible SEP file patches to SAVCE box per online instructions

- re-enabled auto-protect, re-enabled default file cache, and disabled nightly scan

- per Paul's recommendation, notified our org's licensing group that an official request for a certified fix needs to be submitted to Symantec

 

-=-=-= sitrep/observations

- box has been running under load without interruption to sharing services for a little over 72 hours
 

hrssepm's picture

I am seeing this on a WIN2K3 Standard R2 32-bit server. i noted several posts above relating this problem, too. Does the hotfix apply to this OS, too? The KB article does not indicate this.

TIA - Jim

hsshelp's picture

Hi Jim - unfortunately, we don't have any Win2003 (32-bit or 64-bit) server boxes in play so I don't have an answer or suggestion.

hrssepm's picture

hsshelp - thanks for your response!

Does anyone else have any experience using the hotfix on 32-bit OS's?

I will be calling BCS this AM and will post what I find out.

hrssepm's picture

I am seeing this on a WIN2K3 Standard R2 32-bit server. i noted several posts above relating this problem, too. Does the hotfix apply to this OS, too? The KB article does not indicate this.

TIA - Jim

 

Update - Spoke with BCS this AM and they confirmed that the fix was intended for Win2K8 64-bit only. Back to the 'run diags and wait' motif. Getting a little worried as this has been around for at least 8 months and still no definitive fix. Ticket # is 230-645-406 for any interested parties.

Conestoga Rovers's picture

no hotfix that I found to work for WIN2K3 servers. I tried the quick fix for the 2008 servers and for almost 4 weeks my servers were OK. Suddenly three different servers in three different locations lost the shared drives. Second time when this happens, so I am wondering if the SEPM doesn't have a role in this?

Frank van Braak's picture

Same problem with locking/disabled shared on Win2K3 with an upgraden SEPM 11.0.2 to 11.0.4 (MR4 this is?) here..

Now deactivated AutoProtect in the manager and pushed the policies to all clients again.

 

But this sounds like some kinda honeypot...

 

Can I apply the 2008 downloaded fix on our 2003 Small Business Server (and HOW can I apply this to all clients, without copying the files on all clients by hand..?

 

Thanks so far.

Conestoga Rovers's picture

you apply the fix to servers only.
It was easier for me to modify a script found on an earlier post. Create a folder 2008_32-bit_fix. Copy inside the 5 files from Symantec plus the batch file.
For the batch make a new txt file, cope and paste the following:

@ECHO OFF
CLS
SETLOCAL

REM **Instructions**
REM Set SOURCEDIR to the path of the hotfix files
REM No trailing backslash, no quotes
SET SOURCEDIR=C:\2008_32-bit_fix

REM Preflight
IF NOT EXIST "%SOURCEDIR%\SavRT32.dll" GOTO ERR_SOURCEDIR
IF EXIST "C:\Program Files\Common Files\Symantec Shared\SRTSP\2008_32-BIT_FIX.TXT" GOTO ERR_ALREADYRUN

NET STOP "Symantec Endpoint Protection"

REM Back up the old files in Common Files
CD C:\Program Files\Common Files\Symantec Shared\SRTSP
FOR %%A IN (SavRT32.dll Srtsp32.dll) DO REN %%A %%A.bak

REM and replace them
COPY "%SOURCEDIR%\*.dll"

REM Same for Drivers
CD "%SystemRoot%\System32\Drivers"
FOR %%A IN (srtsp.sys srtspl.sys srtspx.sys) DO REN %%A %%A.bak
COPY "%SOURCEDIR%\*.sys"

REM Set a flag so we know that we ran already, in case we run it again by mistake
ECHO.
ECHO SEP 11 32-bit hotfix 1/30/2009 has been applied. > "%ProgramFiles%\Common Files\Symantec Shared\SRTSP\2008_32-BIT_FIX.TXT"

NET START "Symantec Endpoint Protection"
IF NOT %ERRORLEVEL%==0 GOTO ERR_SERVICE

ECHO Hotfix has been applied.  Restart the computer.

GOTO END

:ERR_SOURCEDIR
ECHO Edit the script to set the value of SOURCEDIR to the path to the hotfix files.
GOTO END

:ERR_ARCH
ECHO.
ECHO This hotfix is for x64 machines only.
GOTO END

:ERR_SERVICE
ECHO.
ECHO Abnormal Termination: Symantec Endpoint Protection service encountered a
ECHO problem when starting.  Check Event Log for more information.
GOTO END

:ERR_ALREADYRUN
ECHO.
ECHO This hotfix has already been applied.
GOTO END

:END
ECHO.
PAUSE
ENDLOCAL

Rename .txt to .bat.
I copy this folder to the root of each server I have to patch and run the script from there.

That wasn't enough. This fix worked for 4 weeks only. Last week I was told to disable the File Cache from File System Auto Protect \ Advanced
 

Frank van Braak's picture

Thanks for clearing this... But hopefully Symantec will upgrade there software soon, so this problem wil be gone.

Ozu's picture

 https://forums2.symantec.com/syment/view_profile?user.id=110497
see this link can help you out..!!

Conestoga Rovers's picture

Ozu,
what link?
I have an open case with Symantec for so long and still no answer. They are trying to get a full memory dump when my server is stopping sharing the network drives but I never could get a proper one.

spasq's picture

I am on MR4 and am seeing as similar issue. Should this issue have been fixed in this release (or previous release) or is there a separate patch independant of any release that I need to install. Basically if I install Endpoint 11.4 on a 64 bit Windows 2008 server it pretty much instantly hoses up the box. From the console it runs pretty slow and from an RDP session it locks up and disconnects almost instantly.  If there is a patch where can I get it?

spasq's picture

I should add that as soon as in uninstall Endpoint and reboot the server run without issue.

Frank van Braak's picture

Yes, tried al this. But this error doesn't only occure on Windows Server 2008, but also on Windows 2003 Small Business Server.

But the sollutions doesn't work eighter.

Conestoga Rovers's picture

That solution works for Windows 2008 32bit. (look for the 32bit fix). All my Windows 2008 stopped crashing since I used it. They said at that time that we can try it on W2K3 but wasn't certified for it.
It helps but still, every 4 weeks my servers stop sharing the network drives and I have to reboot them.
Now Symantec waits for me to generate a full memory dump when this happens. Unfortunately my two busy production servers don't have enough free space on the C drive and the dump files are corrupted.
Anybody who wants to work with me into getting these dumps, feel free to contact me. Only submitting these memory dumps from W2K3 servers Symantec can understand what is happening at the moment of crash.

Conestoga Rovers's picture

Paul Murgatroyd,
we did not find a working solution yet for our Windows 2003 environment. I used the Windows 2008 hotfix, it is working for Win2008 servers, it made the 2003 servers more stable than without it but we still have the same issues.
The server doesn't lockup every 3-5 days, it can be up for 4 weeks or more, but still once in a while we are loosing our network drives.

Frank van Braak's picture

Same here... Still not fixed for Windows Small Business Server 2003. (applied all the patched and fixes so far.)

Anyone an sollution...

Conestoga Rovers's picture

no solution.
Frank, I have an opened ticket with Symantec and they are waiting a memory dump from me. Two of my production servers are low in C drive space and I cannot get the mem dump the way I was told.
Symantec is telling us that no one else called them and they don't have a memory dump as they could not replicate our problem.
How often is your system halting? Do you think you can help us with the memory dump? Everybody is going to benefit if we get it right.

Frank van Braak's picture

Sometimes it occures weekly, sometimes it's going correctly for lets say 3 weeks.

Maybe you can explain on how to make that memory dump?

Conestoga Rovers's picture

sure,
I sent you a Private Message with my e-mail address. Anybody interested in getting a memory dump and submitting it to Symantec for a final solution for Win2003 servers is more than welcome to contact me.

cooperriis's picture

I know this post has been going on for a long time and I need some help.

Not long after migrating our system to Server 2008 x64 as a domain controller, I tried a trial install of SEP (not sure of the version number, but it was probably before any of the MR came out) ( I know, BIG mistake.)

Anyway, immediately tried to uninstalled it as it caused a multitude of problems. The application uninstall failed badly. I finally had to manually go through the registry and delete hundreds of registry keys.

This has still apparently left some symantec stuff behind as we have been experiencing the classic network loss that everyone on this thread is experiencing, i.e. network shares disappearing/locking up / sysvol unreachable/ DFS broken, etc. The only thing that will fix the problem temporarily is a server reboot.

All of this accompanied by group policy errors stating that Group Policies can't process (sysvol can't be reached). The 1068 / error code 64 in event viewer that is tracable back to symantec SEP.

I have tried to download and install the SEP MR4 in hopes it would fix the problem. Instead it prevented all users from logging and turned web pages on the server to gibberish.

Well hoping that a clean uninstall might remove what might have been left from before, I tried that. No dice.

This was occuring every week or so but has become more common and on Monday had to restart the server 2008 3 times to keep people connected.

Since we are a non-profit we get donated symantec software for an administrative fee. This does not include any technical support except the forum.

Can someone PLEASE render some assistance of a patch, a tool to clean all traces of SEP or some other information?

We are cuyrrently running SAV 10 Corp on the Server and as well as BUPExec Client. I will gladly reload those instead of completely wiping and reloading the domain controller.

Can someone from symantec please help? Thank you.

**** EDIT  **
On re-reading the forum, I see that SAV MAY have the same issues on Server 2k08 x64... Is that correct?
We are running a managed vista x64 SAV client on the server 2k08 as nothing else will work on it. (Where is x64 for 2k08?)
The version is SAV Corporate Ver. 10.2.0.298

Could this be the culprit???

Will disabling auto-protect fix the issue or is there a patch for this also?

Thanks for any information. A one-man IT department does not have time for this... :-)

jrudbecka's picture

Hi

If you contact support, they can give you a tool called cleanwipe, it will remove all symantec products from your server.

When you install on your server, only install the antivirus part.

Don't think they have fixed SAV 10.2 yet, we are also running a 10.x envorioment, but had to installed SEP to support our 2008 x64 servers.

cooperriis's picture

I thought this was support.
This web forum is the only support we get from symantec since we are a non-profit.
I have no way of contacting support other than this without a service contract

I don't necessarily want to remove all Symantec products. I just want to fix the loss of the network to my users on a regular basis.

Can a Symnatec employee point me in the direction of what I need to do?

Being a non-profit, we may not have to pay full retail for your products, but we still pay for them and spend a good deal of $$ yearly for our various locations.

jrudbecka's picture

Oh yes, you can get some kind of support inhere, most of the times it is good.

If you what to fix your server, you should start doing this by running the cleanwipe tool, so you get the old installation of the server.

The reinstall the latest version, mr4mp2. This should fix the connections problems.

But don't install the firewall on your DC. only AV.

cooperriis's picture

Thanks for the reply. From what I understand, the "CleanWipe" tool is only available if symantec gives you access to it. I guess whithout their access I have to find my own on the internet.
We don't even want to run the SEP but will stick with SAV if that is fixable. If not it is time to explore other AV clients for our Srvr 2008 and possibly the rest of the organization servers and desktops/laptops. If a problem has been festering for over a year without a suitable resolution, that makes me wonder. Thanks for your responses and information.

Mark

OBTW, had auto-protect disabled today. No problems today. Ran a manual scan just now of the domain and DFS and BOOM, group policy errors and network hangs.
What a mess

RandyC's picture

I'm having this problem with the latest 10.2.2.2000......  I can't apply the KB from SEP because I don't have the licensing for SEP.........  SO i'm really stuck here!!!!!!!!!!  I need an update for SAV 10.2 desparately.  Thanks!!!!!!!

update:  I'm on the phone with Symantec Tech. Support and he's telling me I must purchase an upgrade to SEP....  I still have a contract here for SAV 10.2.2 which is still active and I don't want to have to spend the money on an upgrade!!!!   With regards to Paul's post that the AP code is the same so I shoudl be able to apply the update for SEP to SAV, how do i get the update?  It's not listed as an available download (because I don't own a license for SEP, I'm sure)... So is there another way to get the patch?  Can someone link me to an FTP site, or can Symantec make it available for download or send it to me?  This is a major problem!  Thanks

jrudbecka's picture

Hi RandyC

It should have been fixed in 10.2.2, but it doesn't work, this "fix" liste is also from before they released the latest fix for Mr4 mp1, so I think you have to get a SEP11 Mr4 Mp2 client if you what to run symantec on your server. :-((   ("#%/&%""#& -- I know!! :-) )

From SAV 10.2.2.2000's release notes.

Windows 2008 dropping network shares with AutoProtect enabled
Fix ID: 1296949
Symptoms: Network shares become unresponsive after installing Symantec Endpoint Protection MR2 with AutoProtect enabled on a Windows 2008 server.
Solution: Modified Auto-Protect to address the problem.