Vidéos d'aide de Screencast
Symantec to Separate Into Two Focused, Industry-Leading Technology Companies. Learn more.

Full server recovery EV 10

Created: 03 Décembre 2012 • Updated: 05 Mai 2013 | 19 comments
Ce problème a été résolu. Voir la solution.

I'm trying to perform a full disaster recovery test of my Exchange / EV environment.  Running Exchange 2010 and EV 10.  The Exchange recovery works without issue.  The integration with EV appears to function - retrieving items is working, the journal archive appears to be working.  But there's something not right with the journal indexing.  I'm not able to search for new journaled items and the archive vault stops at the date of the backup. 

The backup is done with Dell Appassure Reply which takes a snapshot.  The SQL database and the Evault server are backed up at the same time while the stores are in backup mode.

Commentaires CommentairesAccéder au dernier commentaire

l'image des Rob.Wilcox

Silly questions maybe, but ...

After your test recovery..  are items being picked up from the journal mailbox?  Is the indexing service running?  Are the indexes and journal vault stores both back out of backup-mode?

l'image des JesusWept3

You may also want to run a synchronization against the journal indexes anyway which is good practice after a DR scenario as well

But agreed with rob, sounds like its just in a read only state

l'image des MikeM-2468

It makes sense that it would still be in backup mode after the restore.  I just confirmed that it is out of backup mode.  How long should it take for the new items to show up considering there has been no email at all since the backup.  How is the synchronization performed? 

l'image des Rob.Wilcox

If there has been no email since the backup, then nothing will be in the journal mailbox, and therefore nothing new will be added to the journal archive.

Or am I missing something there?

EV10 index synchronisation is done from the 'Manage Indexes' tool, which you get to by right clicking 'Indexes' in the VAC.

l'image des MikeM-2468

Sorry, I wasn't clear.  There was no mail since the backup except my test emails today.

l'image des MikeM-2468

I just did a synchronize and it took about a minute to run.  Still don't see the new items and the date still ends at the backup time. 

l'image des Rob.Wilcox

Where have you done the restore/recovery to?

Is journaling on the Exchange-side of things configured in that environment?

If it isn't then sending/receiving a few test emails won't add items to the journal archive.. since there is nothing being journaled by Exchange.

If journaling is enabled on Exchange, then have a look at the journal mailbox.  Are there any items in it?

l'image des MikeM-2468

It's a full recovery environment.  All of Exchange and Evault is present.  Items hit the journal mailbox, disappear but never show up in the search. 

l'image des MikeM-2468

In the log I see:

Index volume is marked as failed.

Archive Name: Journal Archive
Index Volume ID: 1956A8B1D0F8AC24EAE14D8BDF7AE99471110000archive.domain.com_475
Reason: None
Error Type: FrenzyErrorDetected

l'image des Rob.Wilcox

Looking in Google-land, the frenzyerrordetected coule be solved by:

http://www.symantec.com/business/support/index?page=content&id=TECH196901

However I would expect your production journal archive would be suffering this same problem - simply doing a restore of the environment isn't going to cause the index to be rebuilt.... unless you issued a rebuild request?

Have a look at the Journal Archive index..  is it marked as failed?

The event you referenced above, when was it logged in relation to:

* The backup

* The restore

* The synchronise

* Ingesting new items (your test emails)

l'image des ia01

Rob, URL mentioned in your post says wink

Solution

This issue will be fixed in Enterprise Vault 10.0.3 **not yet released**

http://www.symantec.com/business/support/index?pag...

l'image des MikeM-2468

The journal archive index is marked as failed.  The event occurred during the synchronize.  But the synchronize said it was successful.

I think it must be some sort of corruption.  The restore process must be flawed.  Restoring 4TB seems to be a challenge.  I notice that when it reboots, it wants to run chkdsk.  I let it run the last time. 

Is a rebuild worth trying?

l'image des JesusWept3

Don't rebuild, you'll just run in to the same errors
The frenzy being detected means that it had 10,000 errors in 7200 seconds, so you need to find out what those errors are and resolve them.

Its most likely something like path not found for the source DVS files or something like that, once you determine what the errors are and correct them, then resynchronize..but again a rebuild will just give you the same issue

l'image des MikeM-2468

Any way to narrow it down?   The example errors I see specify a CAB file.  Mine doesn't.  Mine is also showing errors 1-25.

l'image des MikeM-2468

Verifying the index volume comes back successful.  I'm not able to verify the index location - it's greyed out.

l'image des JesusWept3

OK so the questions would be:
What exact version of EV 10 are you running? (Base? SP1? SP2?)
Have you ever been on EV2007 or earlier or did you start with EV8 or above?

I think we'd need to see some actual example errors from your logs to see what is actually going on.
You can test a full dump through EVSVR to see if that helps retrieve the items or narrow the issue down further, my best guess is that its most likely pointing to an invalid path somewhere

So like if you do
USE EnterpriseVaultDirectory
SELECT * FROM PartitionEntry

The partition paths specified there, are they correct and do they exist?
Or did drive letters get changed or were they restored to a slightly different place etc?

l'image des MikeM-2468

I think the restore was not done properly.  It appears that the SQL database was newer than the EV storage partitions.  I need to get a restore from the same point in time for both.

l'image des MikeM-2468

I finally got a good restore with both SQL and the EV data from the same point in time while everything was in backup mode.  The current journal index is still showing failed, but I can search up to the point of the backup.

The server is 10.0.1.1171.  It was upgraded many times from the old KVS versions through current.  I believe it was on 2007 at one point. 

All the partition paths are present and in the correct place.  The restore was a full snapshot - everything is identical.

Running EVSVR verify on the index comes up with many errors like:
2013-03-22 21:50:42                      Event Output: Unable to complete retrieval request      Reason:           The system cannot find the path specified.  (0x80070003)   Vault:              12 13 2012 to   Vault Id:           10F1CB019EF2ECE48A220E7F980872F251110000archive.domain.com   Item Id:            201303213854289~201303210821230000~Z~50DB6BC227898B02E2138DD7F731FAD1   Reference:          Vault/RI   Extended Item Info:    Supplementary Info:                                (0xc0041ae2)

I remember when I was on 9, I was told by Symantec there was a way to recover the index if it was completely lost.  Is that an option?  Since this discussion was begun I cut a new journal index so that it would be smaller and easier to recover.

SOLUTION