Full server recovery EV 10
Created: 03 Dec 2012 | Updated: 05 May 2013 | 19 comments
This issue has been solved. See solution.
I'm trying to perform a full disaster recovery test of my Exchange / EV environment. Running Exchange 2010 and EV 10. The Exchange recovery works without issue. The integration with EV appears to function - retrieving items is working, the journal archive appears to be working. But there's something not right with the journal indexing. I'm not able to search for new journaled items and the archive vault stops at the date of the backup.
The backup is done with Dell Appassure Reply which takes a snapshot. The SQL database and the Evault server are backed up at the same time while the stores are in backup mode.
Discussion Filed Under:
Comments 19 Comments • Jump to latest comment
Silly questions maybe, but ...
After your test recovery.. are items being picked up from the journal mailbox? Is the indexing service running? Are the indexes and journal vault stores both back out of backup-mode?
Many Thanks,
Rob
www.quadrotech-it.com - All your EV Tools
PS I hope that the post proves helpful.
You may also want to run a synchronization against the journal indexes anyway which is good practice after a DR scenario as well
But agreed with rob, sounds like its just in a read only state
It makes sense that it would still be in backup mode after the restore. I just confirmed that it is out of backup mode. How long should it take for the new items to show up considering there has been no email at all since the backup. How is the synchronization performed?
If there has been no email since the backup, then nothing will be in the journal mailbox, and therefore nothing new will be added to the journal archive.
Or am I missing something there?
EV10 index synchronisation is done from the 'Manage Indexes' tool, which you get to by right clicking 'Indexes' in the VAC.
Many Thanks,
Rob
www.quadrotech-it.com - All your EV Tools
PS I hope that the post proves helpful.
Sorry, I wasn't clear. There was no mail since the backup except my test emails today.
I just did a synchronize and it took about a minute to run. Still don't see the new items and the date still ends at the backup time.
Where have you done the restore/recovery to?
Is journaling on the Exchange-side of things configured in that environment?
If it isn't then sending/receiving a few test emails won't add items to the journal archive.. since there is nothing being journaled by Exchange.
If journaling is enabled on Exchange, then have a look at the journal mailbox. Are there any items in it?
Many Thanks,
Rob
www.quadrotech-it.com - All your EV Tools
PS I hope that the post proves helpful.
It's a full recovery environment. All of Exchange and Evault is present. Items hit the journal mailbox, disappear but never show up in the search.
In the log I see:
Index volume is marked as failed.
Archive Name: Journal Archive
Index Volume ID: 1956A8B1D0F8AC24EAE14D8BDF7AE99471110000archive.domain.com_475
Reason: None
Error Type: FrenzyErrorDetected
Looking in Google-land, the frenzyerrordetected coule be solved by:
http://www.symantec.com/business/support/index?page=content&id=TECH196901
However I would expect your production journal archive would be suffering this same problem - simply doing a restore of the environment isn't going to cause the index to be rebuilt.... unless you issued a rebuild request?
Have a look at the Journal Archive index.. is it marked as failed?
The event you referenced above, when was it logged in relation to:
* The backup
* The restore
* The synchronise
* Ingesting new items (your test emails)
Many Thanks,
Rob
www.quadrotech-it.com - All your EV Tools
PS I hope that the post proves helpful.
Rob, URL mentioned in your post says
Solution
This issue will be fixed in Enterprise Vault 10.0.3 **not yet released**
http://www.symantec.com/business/support/index?pag...
Yep - that's right.
Many Thanks,
Rob
www.quadrotech-it.com - All your EV Tools
PS I hope that the post proves helpful.
The journal archive index is marked as failed. The event occurred during the synchronize. But the synchronize said it was successful.
I think it must be some sort of corruption. The restore process must be flawed. Restoring 4TB seems to be a challenge. I notice that when it reboots, it wants to run chkdsk. I let it run the last time.
Is a rebuild worth trying?
Don't rebuild, you'll just run in to the same errors
The frenzy being detected means that it had 10,000 errors in 7200 seconds, so you need to find out what those errors are and resolve them.
Its most likely something like path not found for the source DVS files or something like that, once you determine what the errors are and correct them, then resynchronize..but again a rebuild will just give you the same issue
Any way to narrow it down? The example errors I see specify a CAB file. Mine doesn't. Mine is also showing errors 1-25.
Verifying the index volume comes back successful. I'm not able to verify the index location - it's greyed out.
OK so the questions would be:
What exact version of EV 10 are you running? (Base? SP1? SP2?)
Have you ever been on EV2007 or earlier or did you start with EV8 or above?
I think we'd need to see some actual example errors from your logs to see what is actually going on.
You can test a full dump through EVSVR to see if that helps retrieve the items or narrow the issue down further, my best guess is that its most likely pointing to an invalid path somewhere
So like if you do
USE EnterpriseVaultDirectory
SELECT * FROM PartitionEntry
The partition paths specified there, are they correct and do they exist?
Or did drive letters get changed or were they restored to a slightly different place etc?
I think the restore was not done properly. It appears that the SQL database was newer than the EV storage partitions. I need to get a restore from the same point in time for both.
I finally got a good restore with both SQL and the EV data from the same point in time while everything was in backup mode. The current journal index is still showing failed, but I can search up to the point of the backup.
The server is 10.0.1.1171. It was upgraded many times from the old KVS versions through current. I believe it was on 2007 at one point.
All the partition paths are present and in the correct place. The restore was a full snapshot - everything is identical.
Running EVSVR verify on the index comes up with many errors like:
2013-03-22 21:50:42 Event Output: Unable to complete retrieval request Reason: The system cannot find the path specified. (0x80070003) Vault: 12 13 2012 to Vault Id: 10F1CB019EF2ECE48A220E7F980872F251110000archive.domain.com Item Id: 201303213854289~201303210821230000~Z~50DB6BC227898B02E2138DD7F731FAD1 Reference: Vault/RI Extended Item Info: Supplementary Info: (0xc0041ae2)
I remember when I was on 9, I was told by Symantec there was a way to recover the index if it was completely lost. Is that an option? Since this discussion was begun I cut a new journal index so that it would be smaller and easier to recover.
Would you like to reply?
Login or Register to post your comment.