Video Screencast Help
Symantec to Separate Into Two Focused, Industry-Leading Technology Companies. Learn more.

Rman database duplication failed on same server with different oracle home.

Created: 10 Mar 2010 • Updated: 07 Mar 2013 | 15 comments
This issue has been solved. See solution.

Hi,

We have netbackup 6.5 server software installed on windows 2003  machine and netbackup 6.5 client + oracle agent installed on unix 11.23 v2 .

On unix client we are able to take database rman backup on tape & restore in same database.

But when we are tring to duplicate database with other instance (which is on same server but in different oracle home) using rman. We are getting  following error.
========================================================
ORA-19870: error reading backup piece dfcon_DEV_5ql87v56_186_1.bak
ORA-19507: failed to retrieve sequential file, handle="dfcon_DEV_5ql87v56_186_1.bak", parms=""
ORA-27029: skgfrtrv: sbtrestore returned error
ORA-19511: Error received from media manager layer, error text:
   Backup file <dfcon_DEV_5ql87v56_186_1.bak> not found in NetBackup catalog
=========================================================

Please note since it is on same server we have software installed for netbackup client & oracle agent. We have done "oracle linking with veritas" for new oracle home.

Please advice,
Mekkalki.

Comments 15 CommentsJump to latest comment

Marianne's picture

Please post dbclient log on client.
If folder does does exist, create it followed by chmod 777 dbclient.
Retry the restore and post log.

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

Marianne's picture

One more thing - are you 100% this backup image (dfcon_DEV_5ql87v56_186_1.bak) is a NetBackup image and not a backup to disk? The filename does not look like an NBU image.
Confirm rman images on master from cmd:
Example from manual: The following example uses bplist to search all Oracle backups for a client named jupiter:
...\netbackup\bin\bplist -C jupiter -t 4 -R /
/exb_n2bm5bco_1_1392342936
/exb_mabm02ko_1_1392170136
/exb_lqbltds6_1_1392083334

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

Mekkalki's picture

Hi,

Thanks for update.

On netbackup server we have dbclient folder at C:\Program Files\Veritas\NetBackup\logs\dbclient location we gave all access to dbclient folder and ran our rman scripts on unix client. But it is not generating any logs in dbclient folder.

On unix client we dont have any dbclient folder. If its required on unix client for collecting the logs please let us know the exact location where we need to create dbclient  folder.

Please advice,

Mekkalki. 

Mekkalki's picture

Hi,

"dfcon_DEV_5ql87v56_186_1.bak" we gave this name to datafile backup in our rman script.

In rman catalog we are able to see this file name & it is showing us as available in rman catalog as follows.
-------------------------------------------------------------------------------------------------------
BS Key  Type LV Size       Device Type Elapsed Time Completion Time
------- ---- -- ---------- ----------- ------------ ---------------
22003   Incr 0  217.81G    SBT_TAPE    01:21:09     10-MAR-10
        BP Key: 22006   Status: AVAILABLE  Compressed: NO  Tag: DEV_DAILY_1ST_WED_2
        Handle: dfcon_DEV_5ql87v56_186_1.bak   Media:
  List of Datafiles in backup set 22003
-------------------------------------------------------------------------------------------------------

We ran bplist command given by you on our unix client. But it is not returning anything. Please see below command.
-------------------------------------------------------------------------------------------------------
orarec@kdddev2:/usr/openv/netbackup/bin>bplist -C kdddev2 -t 4 -R
EXIT STATUS 227: no entity was found
-------------------------------------------------------------------------------------------------------

What backup we are taking on tape is not valid ? or it not proper way to take backup on tape ? or it is not recorded in netbackup catalog ?

Please advice.
Mekkalki. 

Mekkalki's picture

We are taking rman backup from unix client through custom script which we made as per our requirement. In script we are allocating channels of device type sbt_tape. Then we are taking datafile +archive log backup &controlfile backup on tape.In rman catalog we are able to see all backups as avalible backup and netbackup server also shows tape backup completed successfully without error.

Please advice.
Mekkalki. 

Marianne's picture

Extract from NBU for Oracle manual:
To enable the database agent logs
1 Create the following directories on the client:
...
/usr/openv/netbackup/logs/bpdbsbora
/usr/openv/netbackup/logs/bphdb
/usr/openv/netbackup/logs/bporaexp
/usr/openv/netbackup/logs/bporaimp
...
/usr/openv/netbackup/logs/dbclient
For example:
cd /usr/openv/netbackup/logs
mkdir bphdb
2 For 64-bit, also create the following directories
/usr/openv/netbackup/logs/bporaexp64
/usr/openv/netbackup/logs/bporaimp64
3 Set the access permissions to 777 on these log directories.
For example:
chmod 777 bphdb

Try to add start and end date to bplist command. I think the default is last 24 hours:

-s mm/dd/yyyy  -e mm/dd/yyyy

DO NOT FORGET  the "/" after -R:
-R /

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

Mekkalki's picture

Hi,

NBU images on master (using bplist) before attempting another restore

=======================================================================================================
C:\Program Files\Veritas\NetBackup\bin>bplist -C kdddev2 -t 4 -R /
cfc-3787241056-20100310-01_DEV:\
5sl884j3_1_1:\
cfc-3787241056-20100310-00_DEV:\
dfcon_DEV_5pl87v55_185_1.bak:\
dfcon_DEV_5ql87v56_186_1.bak:\
cfc-3787241056-20100309-02_DEV:\
cfc-3787241056-20100309-01_DEV:\
cfc-3787241056-20100309-00_DEV:\
5kl84s9c_1_2:\
5hl84gbo_1_2:\
5il84gbp_1_2:\
cfc-3787241056-20100308-02_DEV:\
5cl81pbd_1_2:\
5bl81pbc_1_2:\
cfc-3787241056-20100308-01_DEV:\
cfc-3787241056-20100308-00_DEV:\
5el8252q_1_2:\

C:\Program Files\Veritas\NetBackup\bin>
=======================================================================================================

NBU images on client (using bplist) before attempting another restore

=======================================================================================================
orarec@kdddev2:/usr/openv/netbackup/bin>bplist -C kdddev2 -t 4 -R /
EXIT STATUS 227: no entity was found
=======================================================================================================

We have created folders on client & given 777 permissions to the folders as suggested. Now we are going run restore.

Please advice.
Mekkalki.
 

Mekkalki's picture

Hi,

We ran restore one more time after proper folders creation but except dbclient folder nothing else is got populated with log file.

We are uploding log file generated in dbclient folder at client side.
We made change in log file as client ip address with x.xx.xxx.xxx and master ip address with y.yy.yyy.yyy nothing else is changed.

Please advice.
Mekkalki.

AttachmentSize
dbclient_log.zip 2.95 KB
Marianne's picture

Problem in dbclient log:
Backup file <5cl81pbd_1_2> not found in NetBackup catalog

I can see in the dbclient log that the client name is kdddev2 - the same name that was used for the backup (output of bplist).
I can also see that the required image 5cl81pbd_1_2 does indeed show up in the bplist output:
5cl81pbd_1_2:\

The error 'not found in NetBackup catalog' was returned by bprd & bpdbm on the master server. We now need to see the bprd log on the master server for the same period (12:16 - 12:17). This will tell us how the master server is interpreting the restore request.

Please find and post the section in bprd log from the time that the incoming request is received from client's IP address.

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

Mekkalki's picture

Hi,

kdddev2 is the client from where we are taking database backup from one instance & on same server but different oracle home where we want restore & create another instance.

We are uploading bprd.log from master for some period.

Same like dbclient log we edited bprd.log. We changed client ip address with x.xx.xxx.xxx and master ip address with y.yy.yyy.yyy

Please advice.
Mekkalki.

AttachmentSize
bprd_1.zip 16.68 KB
Marianne's picture

I don't understand why bplist on master server works fine but not from client. Same status 227 appears in bprd log.

ONE more thing to check, are you attempting the restore as exactly the same user that performed the backup?

See point 5 in this TechNote (ignore the redirected client restore) about oracle user:
http://seer.entsupport.symantec.com/docs/241957.htm

5. The name of the Oracle user on the alternate client has to be the same as on the source client.  (THIS IS VERY IMPORTANT).
Regarding point 5 and having different oracle users:

In the normal Oracle configuration, the Oracle binary has the SetUID attribute (chmod 4751 or 6751).

It is possible is to take off the SetUID and create the same UNIX account as on the "Source Client".  Start Oracle as this "Source" Oracle user and perform the restore in that way. There are several problems to navigate (access to data directories, audit directories
and dump directories),  but it does work (it makes things easier if both users have the same group ID).

After the restore is complete,  the file permissions should be checked (or ownership changed) before restarting the instance as the "Target" Oracle user.

If the user is the same, I'm officially out of ideas. Hopefully someone else on the list has a solution.
If not, your only option is to log a call with Symantec Support. 

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

Mekkalki's picture

Hi,

Sorry, we tried to run bplist command from different oracle user where we want to restore.
Please find below output of bplist command from oracle user where we have taken rman backups
----------------------------------------------------------------------------------------------------------------------------------------------
oradev@kdddev2:/usr/openv/netbackup/bin>bplist -C kdddev2 -t 4 -R /
/cfc-3787241056-20100310-01_DEV
/5sl884j3_1_1
/cfc-3787241056-20100310-00_DEV
/dfcon_DEV_5pl87v55_185_1.bak
/dfcon_DEV_5ql87v56_186_1.bak
/cfc-3787241056-20100309-02_DEV
/cfc-3787241056-20100309-01_DEV
/cfc-3787241056-20100309-00_DEV
/5kl84s9c_1_2
/5hl84gbo_1_2
/5il84gbp_1_2
/cfc-3787241056-20100308-02_DEV
/5cl81pbd_1_2
/5bl81pbc_1_2
/cfc-3787241056-20100308-01_DEV
/cfc-3787241056-20100308-00_DEV
/5el8252q_1_2
oradev@kdddev2:/usr/openv/netbackup/bin>
----------------------------------------------------------------------------------------------------------------------------------------------
If you notice :\ is not there at the end of file name.

ONE more thing to check, are you attempting the restore as exactly the same user that performed the backup?
--- No, oracle users are different.

Is it possible to do with diffierent user ? or oracle users should be same ?

For restore operation we are conecting to rman catlog & target(i.e source database from we have taken backup) database & then auxiliary database (i.e where we want restore database) . All this command we are running from different oracle home where we want to restore.

Is it required to run  rman database duplication command from oracle user where we have taken database backup?

please advice,
Mekkalki.

Marianne's picture

Yes, the oracle user doing the restore has to be the same as the user that performed the backup.

See TechNote mentioned previously:

http://seer.entsupport.symantec.com/docs/241957.htm

5. The name of the Oracle user on the alternate client has to be the same as on the source client.  (THIS IS VERY IMPORTANT).
Regarding point 5 and having different oracle users:

In the normal Oracle configuration, the Oracle binary has the SetUID attribute (chmod 4751 or 6751).

It is possible is to take off the SetUID and create the same UNIX account as on the "Source Client".  Start Oracle as this "Source" Oracle user and perform the restore in that way. There are several problems to navigate (access to data directories, audit directories
and dump directories),  but it does work (it makes things easier if both users have the same group ID).

After the restore is complete,  the file permissions should be checked (or ownership changed) before restarting the instance as the "Target" Oracle user.

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

SOLUTION
Mekkalki's picture

Hi,

Thanks for update.

It seems limitation with RMAN & netbackup that oracle user should be the same for oracle database clonning. because we have number of oracle instances which we are using for different purpose so keeping same user(i.e production server oracle user name for all servers & oracle instances it is not feasible) for all instances. And our requirement is to clone different oracle instances with production database which is having different oracle user.

Is there any way or workaround to do backup from one oracle user & restore database with other oracle user ?

Point 5 from http://seer.entsupport.symantec.com/docs/241957.htm note is confusing is it the workaround to restore as different oracle user ?
or something else that we could not understand.

Please can you explain.

& if it is the workaround for restore as different user please give us steps to do that.

Please advice,
Mekkalki.

Marianne's picture

You will have to give the TechNote to the System Adminitrator on the HP-UX server.
As the restore will have to be done as the same user name that did the backup, the O/S admin will have to grant all the necessary permissions to allow access to files and folders on the taget database.
Once the database can be started as the same user that did the backup, you should be able to perform the restore.
Permissions will have to be changed to the new oracle user after the restore has completed successfully.

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links