Video Screencast Help
Symantec to Separate Into Two Focused, Industry-Leading Technology Companies. Learn more.

NetBackup 7.5 full Catalog Recovery

Created: 11 Feb 2013 • Updated: 13 Feb 2013 | 16 comments
daza's picture
This issue has been solved. See solution.

Hi,

First time performing full Catalog Recovery on Windows Netbackup 7.5 got into an odd issue. Built a new Windows 2003 x64 server in the lab with same SP as in production and installed NetBAckup 7.5.0.1 as in production, difference is in the tape hardware, single T10k, versus a silo in production.

Made a full Catalog backup with a DR file. Load the tape into the drive, added the tape to the CatalogBackup volume pool and successfully completed the Full Catalog Recovery from the Admin console GUI.

As a result – can see all the policies and looks like all the images were restored to …\NetBackup\db\images folder, but nothing else has been restored – NO storage devices, NO media, NO media servers …

So, the question is – what is changed in 7.5 – Do I have to perform any additional steps to recover EMM and NBDB to get the the same configuration as at the production site.

Anybody with NetBackup 7.5 Catalog Recovery ?

Thanks

Comments 16 CommentsJump to latest comment

Marianne's picture

No change in 7.5 - requirements remain the same:
1. Exact same hostname as Prod master.
2. Same NBU version and patch level.
3. Same installation path.

If any of the above is different, it will cause EMM recover to fail.
Please post the Recover#### log. Look in ...NetBackup\logs\user_ops\(user-name)\logs

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

daza's picture

I can confirm that hostname is the same, same patch 7.5.0,1, same installation path d:\veritas

attached is the recovery log.

Thanks Marianne for a fast response.

AttachmentSize
recovery_log.txt 5.05 MB
Marianne's picture

The log file contains no record of Relational database restore.

Did you use the Catalog Recovery wizard in the GUI or cmd (bprecover -wizard) ?

Which option did you choose when you were presented with recovery options?

 

Recover the entire catalog 
Recover the catalog image files and configuration files
 
From the log, it seems as if you chose option 2?
 
You can try and recover Relational database with:
install_path\NetBackup\bin\admincmd\bprecover -r -nbdb

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

daza's picture

Used wizard in the GUI with Recover the entire catalog. Trying another Catalog tape now.

daza's picture

here is the message from the new tape recovery, that shows why the NBDB was not restored. Am I missing something in the Catalog backups?

08:09:40 (9.xxx) WARNING: The following files and directories will not
08:09:40 (9.xxx) be restored because they are not present when the backup
08:09:40 (9.xxx) was done on 2/11/2013 7:03:29 AM. These files and
08:09:40 (9.xxx) directories were either moved or deleted prior to this
08:09:40 (9.xxx) backup, but did exist in a previous backup. If needed,
08:09:40 (9.xxx) they can be restored by doing a normal restore instead
08:09:40 (9.xxx) of doing a true image restore.
08:09:40 (9.xxx) -----------------------------------------------------------

08:09:40 (9.xxx) /D/Veritas/NetBackupDB/staging/BMRDB.db     is not in the true image list. Skipping.
08:09:40 (9.xxx) /D/Veritas/NetBackupDB/staging/BMRDB.log.1     is not in the true image list. Skipping.
08:09:40 (9.xxx) /D/Veritas/NetBackupDB/staging/BMR_DATA.db     is not in the true image list. Skipping.
08:09:40 (9.xxx) /D/Veritas/NetBackupDB/staging/BMR_INDEX.db     is not in the true image list. Skipping.
08:09:40 (9.xxx) /D/Veritas/NetBackupDB/staging/DARS_DATA.db     is not in the true image list. Skipping.
08:09:40 (9.xxx) /D/Veritas/NetBackupDB/staging/DARS_INDEX.db     is not in the true image list. Skipping.
08:09:40 (9.xxx) /D/Veritas/NetBackupDB/staging/DBM_DATA.db     is not in the true image list. Skipping.
08:09:40 (9.xxx) /D/Veritas/NetBackupDB/staging/DBM_INDEX.db     is not in the true image list. Skipping.
08:09:40 (9.xxx) /D/Veritas/NetBackupDB/staging/EMM_DATA.db     is not in the true image list. Skipping.
08:09:40 (9.xxx) /D/Veritas/NetBackupDB/staging/EMM_INDEX.db     is not in the true image list. Skipping.
08:09:40 (9.xxx) /D/Veritas/NetBackupDB/staging/JOBD_DATA.db     is not in the true image list. Skipping.
08:09:40 (9.xxx) /D/Veritas/NetBackupDB/staging/NBAZDB.db     is not in the true image list. Skipping.
08:09:40 (9.xxx) /D/Veritas/NetBackupDB/staging/NBAZDB.log.1     is not in the true image list. Skipping.
08:09:40 (9.xxx) /D/Veritas/NetBackupDB/staging/NBDB.db     is not in the true image list. Skipping.
08:09:40 (9.xxx) /D/Veritas/NetBackupDB/staging/NBDB.log.1     is not in the true image list. Skipping.
08:09:40 (9.xxx) /D/Veritas/NetBackupDB/staging/SEARCH_DATA.db     is not in the true image list. Skipping.
08:09:40 (9.xxx) /D/Veritas/NetBackupDB/staging/SEARCH_INDEX.db     is not in the true image list. Skipping.
08:09:40 (9.xxx) /D/Veritas/NetBackupDB/staging/databases.conf     is not in the true image list. Skipping.
08:09:40 (9.xxx) /D/Veritas/NetBackupDB/staging/server.conf     is not in the true image list. Skipping.
08:09:40 (9.xxx) /D/Veritas/NetBackupDB/staging/vxdbms.conf     is not in the true image list. Skipping.
 

Marianne's picture

Have you checked status of backup done on  2/11/2013 7:03:29 AM?

How many jobs in this backup?
Status of each job?

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

daza's picture

4 Catalog jobs including the parent job. All 4 jobs completed successfully. In the details it show the staging and validating the images.

daza's picture

I am trying to copy the \NetBackupDB\staging folder to the DR master and will try:

bprecover -r -nbdb

 

daza's picture

all files has been disappear from \staging folder

mph999's picture

On the live side, is there any hint of a failure in one of the catalog jobs for the catalog backup.  As you will now, there are 4 jobs, parent, staging, backup of staging and backup of image DB.

I think at the moment, the place to look for this issue is during the catalog backup.

Would also be worth running a cat_convert -dump on the .f file, to see for 100% what was backed up.  I am sure you would have noticed a backup failure and mentioned it.

Ideas I have ...

Delete and recreate the catalog backup policy.

Run a new full catalog and see if that recovers.

I can't see that this will make much dofference, but soetimes you only find out things by trying them, and just occassionally, doing something you think has no relevance provides a solution.

Martin

 

Regards,  Martin
 
Setting Logs in NetBackup:
http://www.symantec.com/docs/TECH75805
 
mph999's picture

On the live side, are there any sybolic links that you are using to relocate the /usr/opevn/db or .../db/data dirs ?

Martin

 

Regards,  Martin
 
Setting Logs in NetBackup:
http://www.symantec.com/docs/TECH75805
 
Marianne's picture

@daza - I am curious to know on which master you are looking for staging folder and why?

staging is a temporary phase during backup and restore. As Martin suggested - perform another backup on Prod master and monitor each stream - check Windows explorer while staging phase is running and also when relational database is backed up.

Ensure that you copy the newest DR-file. If I remember correctly, there is a problem with catalog recovery if newer images are found for relational database upon completion of the images restore phase - or not found at all in images.

Complete backup on Prod master followed by recovery on test master.

One more thing: PLEASE upgrade to 7.5.0.4 ASAP - there were some issues with NBU catalogs that were only resolved in 7.5.0.3. 

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

mph999's picture

Good points Marianne - if there are no sym links floating around, then my next idea would be some issue with te DR file, or the wrong DR file being used.

THe suggestion of a new full, would mean there is only the refernece to one backup in the DR file, which simplifies things a little.

Martin

 

Regards,  Martin
 
Setting Logs in NetBackup:
http://www.symantec.com/docs/TECH75805
 
Mark_Solutions's picture

Just one more thought that has caught people out before ... do you have any exclusions set on your live Master, just in case files are being skipped.

To double check on the live Master open up the BAR GUI select the Master as the source client and NBU-Catalog as the policy type and see if all the files actually exist within the backup.

 

Authorised Symantec Consultant

Don't forget to "Mark as Solution" if someones advice has solved your issue - and please bring back the Thumbs Up!!.

daza's picture

Great idea. The Catalog tapes had the 2 stages backed up, but the tapes were owned by another media server, so even I changed the hostname in the DR file - did not help, but after changing the Media Host Override to the DR media server - the Recovery completed successfully with EMM and NBDB recovered.

Thank you all for the help

SOLUTION
mph999's picture

Ahh, the DR environment did not match the 'original' ...

Still, fixed, which is the main thing.

Martin

 

Regards,  Martin
 
Setting Logs in NetBackup:
http://www.symantec.com/docs/TECH75805