Video Screencast Help
Symantec to Separate Into Two Focused, Industry-Leading Technology Companies. Learn more.

Error 41320, But with a timeout

Created: 07 Jun 2013 • Updated: 23 Oct 2013 | 17 comments
Mark_Solutions's picture
This issue has been solved. See solution.

Hi

I have a customer running 10.0.3 HF2 on Windows 2008R2 Std (same error used to happen when it was at 10.0.1 by the looks of it)

The index admin service reports event 41320 with the text as follows:

A periodic Index Admin check could not determine the status of one or more Index Volumes that are managed by this server. (Error: Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding.)

For more information, see Help and Support Center at http://entced.symantec.com/entt?product=ev&language=english&version=10.0.3.0&build=10.0.3.1090&error=V-437-41320

Have not been able to find anything about this one on here or the Support site other than the SSPI error type (and the SQL Table is already FQDN so that is OK anyway)

Do I need to dtrace the indexing service (presumably during an indexing service re-start) or is there somewhere else that this may be logged / reported to tell me exactly where the error is and maybe i need to adjust the timeout?

Other relevant information :

Single EV Server (SQL server is a seperate machine), Exchange Mailbox and Journaling from Exchange 2003 and 2010.

Large searches of the journal archives also timeout - event id 41315 - A search failed with the error "The Search did not complete in the required time" - the details of this indicate the timeout is set to 120.

The EV Server also experiences two other issues - event 13360 - error accessing VaultDatabase - details relate to a stored proceedure uspd_Journals having a deadlock on lock resources with another process followed immediately by event 6796 - A COM Exception 0x8040e14 - details relate to ClearExpiredJournalRecords

All may be related so thought i had best include those details too

Any help much appreciated

Operating Systems:

Comments 17 CommentsJump to latest comment

JesusWept3's picture

Definitely sounds like SQL as that is a SQL error

What's the CPU looking like on that SQL server at the moment?

Are you doing SQL maintenance like index rebuilds and such?

I guess you could try increasing the SQL timeout on the SQL server itself but may make things worse

JesusWept3's picture

Oh and yeah dtrace the process that threw the event and maybe a SQL profiler along with a perfmon

I wrote an article about SQL performance troubleshooting here

http://www.symantec.com/connect/articles/troublesh...

Tremaine's picture

If you have already followed all the best practices for SQL maintenance (index rebuild / defrag etc.) without any improvement I suggest you open a support case so we can review this one further.

That being said a change was introduced in 10.0.3 so that the uspd_journals will be chosen as the deadlock victim if necessary to help alleviate any contention with archiving processes. Ideally we need to try and determine why its ending up this way though. Typically that SP does a lot of work especially on busy environments. You could however potentially mitigate some of its impact by ensuring that when coming out of backup mode its during a quiet time of the day as that is when it will normally be executed (or service restart) It also really depends on the size and nature of the content in your JournalDelete/archive etc tables as to how well it will perform.

However as far as the Indexing issue is concerned we really need to understand whether its related as well or something else.

Mark_Solutions's picture

Thanks JW2 - yes SQL is a know issue here - heavily loaded SQL Server and a dedicated SQL server for EV is under consideration

So do you think that would stop the Indexing service assessing the state of an index volume?

The 120 second timeout I have traced to being an IIS timeout - this has been adjusted and advice given about doing smaller searches through the web search.

It is the 41320 error i really need to get to the bottom of - will dtrace the index service during a startup to see what it says.

Authorised Symantec Consultant

Don't forget to "Mark as Solution" if someones advice has solved your issue - and please bring back the Thumbs Up!!.

TheEmptyMind's picture

A periodic Index Admin check could not determine the status of one or more Index Volumes that are managed by this server. (Error: Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding.)

I received this error myself yesterday and fortunately, I was on a call with Symantec support.  Our SQL is not loaded and has plenty of resources and so is EV server.  We bounced Indexing service and we didn't see the issue again.  I suppose that there is some lock on SQL that prevented the indexing service to do it's routine check and I doubt if there is an actual issue with the index volumes, unless of course the event is repeating when you restart the Indexing service.

TheEmptyMind's picture

FWIW, I received the above error again today.  I suspect that it is logging the error about a Index volume that I closed, moved to a different location and updated the IndexRootPathEntry table.  It seems that there may be another reference somewhere in the DB. I will dig further on Monday.

Mark_Solutions's picture

Set up a DTrace during an index re-start - 2 minutes and a 60MB log file - time to open a case to get it examined properly

The errror did ocurr again after the index started up

I will keep the thread updated

Authorised Symantec Consultant

Don't forget to "Mark as Solution" if someones advice has solved your issue - and please bring back the Thumbs Up!!.

Mark_Solutions's picture

Case 04512962 has been opened for this - customer awaiting ftp location to upload the CAB file - appreciate it if anyone in support get this one responded to - Thanks

Authorised Symantec Consultant

Don't forget to "Mark as Solution" if someones advice has solved your issue - and please bring back the Thumbs Up!!.

Simon B.'s picture

Hi Mark,

we are seeing a similar problem at the moment. Did you ever find a solution for your customer or anything that points to the cause of these issues?

 

PMCS GmbH & Co. KG - Consulting und Support für BE/NBU/EV und andere Symantec Produkte.
Please take the time and mark this post as solution if it solved your problem - thanks!

Mark_Solutions's picture

Last I heard Support were still investigating  - I will ask the customer if it was ever resolved (it wasn't just a few weeks back!!)

Authorised Symantec Consultant

Don't forget to "Mark as Solution" if someones advice has solved your issue - and please bring back the Thumbs Up!!.

Mark_Solutions's picture

OK - so no real progress for them yet - they have been asked to follow this tech note and move their page file but it has not resolved anything:

http://www.symantec.com/docs/HOWTO59060

The tech note sounds a bit drastic to me!!

I will keep this updated when i hear any more

Authorised Symantec Consultant

Don't forget to "Mark as Solution" if someones advice has solved your issue - and please bring back the Thumbs Up!!.

Nathan Clark 2's picture

Mark whats you current case #? and have you dtraced IVP IAS?

this looks benign to me, as I suspect its the enqueing of the sync requests timing out > 60 sec (hard coded, probably as the engine is busy)

Mark_Solutions's picture

I think it is 04962295

As it has been dragging on for 4 months now the customer is dealing directly on this so dont have all of the details

Authorised Symantec Consultant

Don't forget to "Mark as Solution" if someones advice has solved your issue - and please bring back the Thumbs Up!!.

Nathan Clark 2's picture

ok thanks have requested an escaltion on it.

Mark_Solutions's picture

Thanks Nathan - will keep the thread updated

Authorised Symantec Consultant

Don't forget to "Mark as Solution" if someones advice has solved your issue - and please bring back the Thumbs Up!!.

modemis's picture

Hello.  Any update to this issue?  We have a customer that is also getting this error.  Thank you.

Mark_Solutions's picture

OK - final update on this one (at last)...

Symantec have closed the case.

They have examined the delta times and can see the occasional "StoragePendingWorkDiscovery.volumeCrawlerRoutine" that times out and is followed immediately by the 41320 error.

They say that the error is periodic and that it recovers as it is using a 60 second timeout

In their words "This is expected behavoir and you can safely ignore there warnings"

I dare say a tech note will appear at sometime - just another worrying event to add to the list not to worry about.

Hope this helps and puts peoples minds at rest if they have a similar issue. Work checking though to ensure it does match issue in the same way.

Authorised Symantec Consultant

Don't forget to "Mark as Solution" if someones advice has solved your issue - and please bring back the Thumbs Up!!.

SOLUTION