Video Screencast Help
Symantec to Separate Into Two Focused, Industry-Leading Technology Companies. Learn more.

SPFILE Autobackup fails after Db moved to SFHA cluster

Created: 29 Sep 2011 • Updated: 17 Oct 2011 | 3 comments
This issue has been solved. See solution.

Last night I migrated my production database to a SFHA cluster made up two Sun T3-1B blade servers.  The nodes of the cluster are: st31bbl01 (Sun T3-1B blade 01) and st31bbl02 (Sun T3-1B blade 02).  The name of the virtual database server than can fail over between these nodes is st31bora01 (Sun T3-1B oracle 01). 

Database backups report as successful within the Netbackup GUI but when I check the log file from the RMAN backup script I see the list of Db files successfully backed up followed by this:

 Starting Control File and SPFILE Autobackup at 29-SEP-11
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03009: failure of Control File and SPFILE Autobackup command on ORA_SBT_TAPE_1 channel at 09/29/2011 01:47:42
ORA-19506: failed to create sequential file, name="PASPROD_c-1968089396-20110929-01", parms=""
ORA-27027: sbtremove2 returned error
ORA-19511: Error received from media manager layer, error text:
   Failed to remove, PASPROD_c-1968089396-20110929-01, from image catalog.

 

When I check the log file under /usr/openv/netbackup/logs/dbclient, I find the following:

 21:46:54.822 [9177] <2> sbtremove2: INF - entering
21:46:54.822 [9177] <2> int_RemoveImage: INF - entering
21:46:54.822 [9177] <2> int_RemoveImage: INF - Removing backup image => <PASPROD_c-1968089396-20110928-04>
21:46:54.822 [9177] <2> xbsa_ValidateFeatureId: INF - entering
21:46:54.822 [9177] <2> xbsa_ValidateFeatureId: INF - leaving (0)

21:46:54.822 [9177] <2> int_FindBackupImage: INF - entering
21:46:54.822 [9177] <2> int_GetBfsDateRange: INF - entering
21:46:54.823 [9177] <2> int_GetBfsDateRange: INF - RMAN file name = PASPROD_c-1968089396-20110928-04
21:46:54.823 [9177] <2> int_GetBfsDateRange: INF - probable auto controlfile
21:46:54.823 [9177] <2> int_GetBfsDateRange: INF - Start range check: bfs_time=1317232800, start=819936000, end=1317354414
21:46:54.823 [9177] <2> int_logDateRange: INF - Start Time = 09/27/11 12:00:00
21:46:54.823 [9177] <2> int_logDateRange: INF - End Time = 09/29/11 12:00:00
21:46:54.823 [9177] <2> int_logDateRange: INF - leaving
21:46:54.823 [9177] <2> xbsa_BeginTransaction: INF - entering
21:46:54.823 [9177] <4> VxBSABeginTxn: INF - entering VxBSABeginTxn.
21:46:54.823 [9177] <4> VxBSAGetEnv: INF - entering GetEnv - NBBSA_DB_TYPE
21:46:54.823 [9177] <4> VxBSAGetEnv: INF - returning - Oracle
21:46:54.823 [9177] <4> VxBSAGetEnv: INF - entering GetEnv - NBBSA_NTUPLE_RESTORE
21:46:54.823 [9177] <4> VxBSAGetEnv: INF - returning -
21:46:54.824 [9177] <4> VxBSAGetEnv: INF - entering GetEnv - NBBSA_COPY_NUMBER
21:46:54.824 [9177] <4> VxBSAGetEnv: INF - returning -
21:46:54.824 [9177] <2> xbsa_BeginTransaction: INF - leaving (0)

21:46:54.824 [9177] <2> xbsa_QueryObject: INF - entering
21:46:54.824 [9177] <4> VxBSAQueryObject: INF - entering QueryObject.
21:46:54.824 [9177] <4> dbc_GetServerClientConfig: entering dbc_GetServerClientConfig.
21:46:54.824 [9177] <4> dbc_GetServerClientConfig: ServerName: <inf-srv17.apacorp.net>, ClientName: <st31bora01>
21:46:54.824 [9177] <4> VxBSAGetEnv: INF - entering GetEnv - BSA_SERVICE_HOST
21:46:54.824 [9177] <4> VxBSAGetEnv: INF - returning - inf-srv17.apacorp.net
21:46:54.824 [9177] <4> VxBSAGetEnv: INF - entering GetEnv - NBBSA_CLIENT_HOST
21:46:54.824 [9177] <4> VxBSAGetEnv: INF - returning - st31bora01
21:46:54.824 [9177] <4> VxBSAGetEnv: INF - entering GetEnv - NBBSA_POLICY
21:46:54.824 [9177] <4> VxBSAGetEnv: INF - returning - ORA_PASPROD
21:46:54.825 [9177] <4> VxBSAGetEnv: INF - entering GetEnv - NBBSA_KEYWORD
21:46:54.825 [9177] <4> VxBSAGetEnv: INF - returning - pasprod
21:46:54.825 [9177] <4> bsa_bplist: entering bsa_bplist
21:46:54.828 [9177] <2> vnet_async_connect: vnet_vnetd.c.3983: connect in progress: 0 0x00000000
21:46:54.865 [9177] <2> vnet_vnetd_service_socket: vnet_vnetd.c.2043: VN_REQUEST_SERVICE_SOCKET: 6 0x00000006
21:46:54.865 [9177] <2> vnet_vnetd_service_socket: vnet_vnetd.c.2057: service: bprd
21:46:55.075 [9177] <2> vnet_async_connect: vnet_vnetd.c.4169: in progress connect: 0 0x00000000
21:46:55.076 [9177] <2> vnet_async_connect: vnet_vnetd.c.4172: connect: async CONNECT FROM 192.168.84.211.63677 TO 192.168.86.11.13724 fd = 14
21:46:55.076 [9177] <2> logconnections: BPRD CONNECT FROM 192.168.84.211.63677 TO 192.168.86.11.13724
21:46:55.076 [9177] <2> vauth_authentication_required: vauth_comm.c.749: no methods for address: no authentication required
21:46:55.076 [9177] <2> vauth_connector: vauth_comm.c.182: no methods for address: no authentication required
21:46:55.076 [9177] <2> bprd_connect: no authentication required
21:46:55.076 [9177] <2> vnet_dlopen_vxss_client_magic: vnet_vxss.c.1766: Assuming no VxSS for DB Agents: 0 0x00000000
21:46:55.076 [9177] <2> bsa_bplist: start_date = Tue Sep 27 12:00:00 2011

21:46:55.077 [9177] <2> bsa_bplist: end_date = Thu Sep 29 12:00:00 2011

21:46:55.077 [9177] <2> bsa_bplist: Request = oraprod dbaprod st31bora01 st31bora01 st31bora01 ORA_PASPROD 7 pasprod 3 999 1317146400 1317319200 4 4 1 1 1 0
4 1017 2007 4 0 C C C C C 0 2 0 0 0
21:46:55.077 [9177] <4> bsa_bplist: Filepath = /PASPROD_c-1968089396-20110928-04
21:46:55.273 [9177] <2> dbc_get_string: Output = EXIT STATUS 131
21:46:55.273 [9177] <16> VxBSAQueryObject: ERR - dbc_get_string() failed 131
21:46:55.273 [9177] <2> xbsa_ProcessError: INF - entering
21:46:55.273 [9177] <2> xbsa_ProcessError: INF - leaving
21:46:55.273 [9177] <16> xbsa_QueryObject: ERR - VxBSAQueryObject: Failed with error:
   Server Status:  client is not validated to use the server
21:46:55.273 [9177] <2> xbsa_QueryObject: INF - leaving (3)

21:46:55.273 [9177] <2> xbsa_EndTransaction: INF - entering
21:46:55.273 [9177] <4> VxBSAEndTxn: INF - entering VxBSAEndTxn.
21:46:55.273 [9177] <4> VxBSAEndTxn: INF - Transaction being COMMITED.
21:46:55.273 [9177] <4> VxBSAGetEnv: INF - entering GetEnv - NBBSA_LOG_DIRECTORY
21:46:55.273 [9177] <4> VxBSAGetEnv: INF - returning - dbclient
21:46:55.273 [9177] <4> VxBSAEndTxn: INF - Cleaning directory: </usr/openv/netbackup/logs/dbclient>
21:46:55.274 [9177] <4> delete_old_files: entering delete_old_files.
21:46:55.274 [9177] <2> xbsa_EndTransaction: INF - leaving (0)

21:46:55.274 [9177] <2> int_FindBackupImage: INF - leaving
21:46:55.274 [9177] <16> int_RemoveImage: ERR - Failed to remove, PASPROD_c-1968089396-20110928-04, from image catalog.

21:46:55.274 [9177] <2> int_RemoveImage: INF - leaving
21:46:55.274 [9177] <2> sbtremove2: INF - leaving
21:46:55.274 [9177] <2> sbterror: INF - entering
21:46:55.274 [9177] <2> sbterror: INF - Error=7501: Failed to remove, PASPROD_c-1968089396-20110928-04, from image catalog.
.
21:46:55.274 [9177] <2> sbterror: INF - leaving 

 

I suspect the message "client is not validated to use the server" indicates that I have a mis-configuration between the node names, the database virtual server name and the cluster name.  Does anyone have any recommendations on where to look to resolve this?

Thanks

Ken

Comments 3 CommentsJump to latest comment

khemmerl's picture

I specified the name of the virtual Oracle DB server (st31bora01) within the NetBackup policy on the Administration Console and updated /usr/openv/netbackup/bp.conf on both nodes of the cluster with the following:

 SERVER = inf-srv17.apacorp.net
CLIENT_NAME = st31bora01
REQUIRED_INTERFACE = st31bora01

This appears to have resolved the issue and autobackups of the DB SPFILEs now runs cleanly.  Nonetheless, I have a ticket open with Symantec support to confirm this is the recommended configuration.

Ken

Ken Hemmerling
Alberta Pensions Services Corporation
Database Administrator
5103 Windermere Blvd. SW
Edmonton, AB T6W 0S9

SOLUTION
Marianne's picture

Double-check the script - specify the Virtual name : NB_ORA_CLIENT=<virtual-name>

See this old but still valid TN: http://www.symantec.com/docs/TECH28412

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

khemmerl's picture

I tried following the paper you recommended and confirmed the NetBackup policy referenced the name of the cluster (st31bora01) and not either of the individual nodes.  I also set /usr/openv/netbackup/bp.conf to reference the actual node name as follows:

SERVER = inf-srv17.apacorp.net
CLIENT_NAME = st31bbl01
REQUIRED_INTERFACE = st31bbl01
VERBOSE = 0
ENABLE_ROBUST_LOGGING = NO 

I set set the database RMAN configuration as follows:

 RMAN> show all;

RMAN configuration parameters are:
CONFIGURE RETENTION POLICY TO REDUNDANCY 14;
CONFIGURE DEFAULT DEVICE TYPE TO 'SBT_TAPE';
CONFIGURE CONTROLFILE AUTOBACKUP ON;
CONFIGURE CONTROLFILE AUTOBACKUP FORMAT FOR DEVICE TYPE 'SBT_TAPE' TO '%d_%F';
CONFIGURE DEVICE TYPE 'SBT_TAPE' PARALLELISM 4 BACKUP TYPE TO BACKUPSET;
CONFIGURE CHANNEL 1 DEVICE TYPE 'SBT_TAPE' MAXOPENFILES 1 SEND
  'NB_ORA_POLICY=ORA_PASPROD, NB_ORA_SCHED=Oracle_Online_Backup NB_ORA_CLIENT=st31bora01';
CONFIGURE CHANNEL 2 DEVICE TYPE 'SBT_TAPE' MAXOPENFILES 1 SEND
  'NB_ORA_POLICY=ORA_PASPROD, NB_ORA_SCHED=Oracle_Online_Backup NB_ORA_CLIENT=st31bora01';
CONFIGURE CHANNEL 3 DEVICE TYPE 'SBT_TAPE' MAXOPENFILES 1 SEND
  'NB_ORA_POLICY=ORA_PASPROD, NB_ORA_SCHED=Oracle_Online_Backup NB_ORA_CLIENT=st31bora01';
CONFIGURE CHANNEL 4 DEVICE TYPE 'SBT_TAPE' MAXOPENFILES 1 SEND
  'NB_ORA_POLICY=ORA_PASPROD, NB_ORA_SCHED=Oracle_Online_Backup NB_ORA_CLIENT=st31bora01';

<default settings removed for brevity> 

 

Unfortunately, a test to run an archive log  backup gives the following error:

 RMAN-03009: failure of backup command on ORA_SBT_TAPE_1 channel at 10/07/2011 12:00:27
ORA-19506: failed to create sequential file, name="PASPROD_al26963_20111007_85945", parms=""
ORA-27028: skgfqcre: sbtbackup returned error
ORA-19511: Error received from media manager layer, error text:
   VxBSACreateObject: Failed with error:
   Server Status:  the specified policy is not of the correct client type
channel ORA_SBT_TAPE_1 disabled, job failed on it will be run on another channel
RMAN-03009: failure of backup command on ORA_SBT_TAPE_2 channel at 10/07/2011 12:00:27
ORA-19506: failed to create sequential file, name="PASPROD_al26957_20111007_85946", parms=""
ORA-27028: skgfqcre: sbtbackup returned error
ORA-19511: Error received from media manager layer, error text:
   VxBSACreateObject: Failed with error:
   Server Status:  the specified policy is not of the correct client type
channel ORA_SBT_TAPE_2 disabled, job failed on it will be run on another channel
RMAN-03009: failure of backup command on ORA_SBT_TAPE_3 channel at 10/07/2011 12:00:27
ORA-19506: failed to create sequential file, name="PASPROD_al26960_20111007_85947", parms=""
ORA-27028: skgfqcre: sbtbackup returned error
ORA-19511: Error received from media manager layer, error text:
   VxBSACreateObject: Failed with error:
   Server Status:  the specified policy is not of the correct client type
channel ORA_SBT_TAPE_3 disabled, job failed on it will be run on another channel
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03009: failure of backup command on ORA_SBT_TAPE_4 channel at 10/07/2011 12:00:27
ORA-19506: failed to create sequential file, name="PASPROD_al26959_20111007_85948", parms=""
ORA-27028: skgfqcre: sbtbackup returned error
ORA-19511: Error received from media manager layer, error text:
   VxBSACreateObject: Failed with error:
   Server Status:  the specified policy is not of the correct client type 

 

Removing the REQUIRED_INTERFACE line from bp.conf not make any difference.  It appears from my testing that TECH28412 is obsolete.  I have put the name of the cluster back in bp.conf.

Ken

Ken Hemmerling
Alberta Pensions Services Corporation
Database Administrator
5103 Windermere Blvd. SW
Edmonton, AB T6W 0S9