Intermittent Oracle Backups Fail - All Oracle Jobs, Other Netbackup Jobs Run Normal
I have an issue with my Oracle DB jobs failing after Netbackup runs for about a week. All Oracle type policies are effected on all Oracle servers. Other Netbackup jobs including MS-SQL run just fine.
In the logs I can see that the client passes the information to bprd and bprd sends it to nbpem, but the child jobs never appear in the activity monitor and the RMAN logs say the following:
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03009: failure of backup command on t1 channel at 03/11/2009 09:16:12
ORA-19506: failed to create sequential file, name="g6k9ks5t_1_1", parms=""
ORA-27028: skgfqcre: sbtbackup returned error
ORA-19511: Error received from media manager layer, error text:
VxBSACreateObject: Failed with error:
Server Status: Communication with the server has not been initiated or the server status has not been retrieved from the server
If I restart the master server, the child jobs appear with a status 50, from the ones that were submitted before the restart of the master and the Oracle backups if run again will work for a few more days and then the cycle repeats.
It almost seems like the job submission to nbpem is failing, even though the bprd log clearly states it submits the child job.
Environment:
Netbackup 6.5.2A on master/media and client
All Solaris 10 hosts on Sparc.
I do not believe this is resolution related,RMAN script related, permission related, filesystem full related, as all it takes is a restart of the master and everything works for awhile again.
Any insight would be appreciated. Note: The Oracle DB's were patched around the time this started to happen.
Regards,
Benjamin Schmaus
Comments
Can this be ruled out?
BUG REPORT: If the execution time of multiple Veritas NetBackup (tm) for Oracle shell scripts overlap on the same physical Oracle host, one or more of the backups may not execute using the expected Oracle backup policy.
http://support.veritas.com/docs/284222
Bob Stump VERITAS - "Ain't it the truth?" Incorrigible punster -- Do not incorrige
Yes, considering the same
Yes, considering the same schedules were in place before we had this issue. Although good attempt :)
I am patching to 6.5.3 this weekend, maybe that will fix. If it does then we call it an nbpem bug.
Regards,
Benjamin Schmaus
consider 6.5.3.1
There are additional bugs in 6.5.3 with nbpem.
Please consider going to 6.5.3.1 as symantec support will recommend it before assisting with problems since there are known problems with 6.5.3 You might as well do them at same time.
I am running 6.5.3.1
Bob Stump VERITAS - "Ain't it the truth?" Incorrigible punster -- Do not incorrige
Create logs direcroty on client/s
create bphdb and dbclient directory under /usr/openv/netbackup/logs on the client/s and set verbose = 5 in client/s bp.conf.
You may be able to find the reason from these logs.
if you go to
if you go to $ORACLE_HOME/RMAN/scripts/log directory find .out file in the same time when backup failed or was in queue..
Bob - That is the plan
Bob - That is the plan currently, to go to 6.5.3.1. Hopefully that will resolve this.
Regards,
Benjamin Schmaus
Hi, I have been facing the
Hi,
I have been facing the same issue. Can you please confirm whether you are using backup network for backups? and is there a entry of required interface in bp.conf on the client?
Is there a backup name for the client or is it the same as production?
Would you like to reply?
Login or Register to post your comment.