Video Screencast Help
Symantec to Separate Into Two Focused, Industry-Leading Technology Companies. Learn more.

NetBackup 6.5.6 not ejecting tapes

Created: 05 Feb 2013 • Updated: 18 Feb 2013 | 16 comments
This issue has been solved. See solution.

I am running Veritas NetBackup 6.5.6 in a mixed evironment with regards to Libraries.  I am using an Oracle SL3000 tape library via an ACSLS controller for the physical offsite creation of tapes.  I am using a DSI VTL emulating an STK L700e tape library to do the day-to-day backups of the various policies.  Essentially I have all of the policies, except the Vaulting job and a Deferred Eject job to handle the physical duplication and ejection of the tapes that are to go offsite.  All drives are LTO4.

The VTL tapes are designated to belong to the Scratch Pool.  The Physical Tapes all assigned to the Duplicate pool.  We are running with a Capacity License module so all modules are operational within the environment.  Vaulting of offsite tapes was working fine until this last weekend.  No changes were made to the Veritas environment prior to the start of the problem.

The Veritas Master server can successfully communicate with the ACSLS server and I can manually eject tapes from the Veritas environment and successfully add tapes back into the environment manually.  The Vaulting job successfully duplicates data from the VTL tapes to the Physical SL3000 LTO4 tapes from the Duplicate pool.

However when I run the deferred Vault Eject policy no tapes are now identified to be ejected.  I have verified that we do not have the 'container' option selected under the Vault Properties and that the correct Robotic Volume Group and Off-Site volume group have been selected/identified.  I have also verified that the Daily Vaulting job is set to include all clients and all locations. 

The duplication job is configured to use the ACSLS Robot (Physical) and the Duplicate Volume Pool.  Additionally the duplication is set to use Removable Media and/or disk.  The Catalog Backup job is enabled and the Eject parameter for the Vaulting job are enabled and configured to vault from the Duplicate volumen pool, eject mode = deferred and immediate suspension of the sessions's media.  E-mail notifications work just fine.  The duplication job is running and completing without any errors.  I can identify the VTL tapes being copied and the SL3000 tapes being written to via the Activity Monitor window just fine.

The ACSLS server has been rebooted, the SL3000 Tape Silo has been rebooted and the Services on the Veritas Master Server for NetBackup have been stopped and restarted and none of this attempts have resolved the issue.  I have also checked for a stale EJECT_0_0_0_0.txt on the Veritas Master Server and did not find a stale file.

Not sure what other information you may need...I need to get Vaulting of the offisite tapes back operational.  Any suggestions you may have are appreciated in advance.

Comments 16 CommentsJump to latest comment

RamNagalla's picture

hi,

what is the backup selection period in choose backup tabe of Vault prfole.?

could you post use the details status of the vault job and the Detail.log and summary.log from the location /usr/openv/netbackup/vault/sessions/<Profilename>/Sessionid>/

and also provide the screenshots of choose backup tab and Eject tape from the vault profile.

BR549's picture

I am using 3 days 0 hours and 0 days 0 hours for the selecting critera. I am assuming you were referring to the Choose Backups tab setting for the Daily_Vault job.

BR549's picture

I have uploaded screenshots of the Daily_Vault & Daily_Eject for the Backup & Eject Tabs for the jobs. I have also uploaded the SIDs that have occurred since the last successful job ran.

ScreenHunter_08 Feb. 05 12.01.gif ScreenHunter_09 Feb. 05 12.01.gif ScreenHunter_10 Feb. 05 12.02.gif ScreenHunter_11 Feb. 05 12.02.gif
AttachmentSize
sid654.zip 328.79 KB
sid655.zip 34.69 KB
RamNagalla's picture

hi,

my first quesiton would be why 2 profiles for same eject task?

what is the output of below command

vmquery -m V10102

BR549's picture

I share the Robotic Arm with a mainframe system and in order to avoid resource contentions I needed to schedule my ejects at a time when the mainframe was not ejecting tapes. Therefore I created a deferred eject policy.

RamNagalla's picture
vltrun@AddSelectedImagesEjectMedia^655: Media=V10105 NOT in Eject Volume Pools, skipping...
03:00:57.111 [14924.14744] <8> vltrun@AddSelectedImagesEjectMedia^655: MDA=V10122 selected but not in RBT_GRP=000_00000_ACS, not ejecting
03:00:57.111 [14924.14744] <2> vltrun@AddSelectedImagesEjectMedia^655: Media=Z00001 NOT in Eject Volume Pools, skipping...
03:00:57.111 [14924.14744] <2> vltrun@AddSelectedImagesEjectMedia^655: Media=Z00003 NOT in Eject Volume Pools, skipping...
03:00:57.111 [14924.14744] <2> vltrun@AddSelectedImagesEjectMedia^655: Media=Z00004 NOT in Eject Volume Pools, skipping...
03:00:57.111 [14924.14744] <2> vltrun@AddSelectedImagesEjectMedia^655: Media=Z00005 NOT in Eject Volume Pools, skipping...
03:00:57.111 [14924.14744] <2> vltrun@AddSelectedImagesEjectMedia^655: Media=Z00013 NOT in Eject Volume Pools, skipping...
03:00:57.111 [14924.14744] <2> vltrun@AddSelectedImagesEjectMedia^655: Media=Z00015 NOT in Eject Volume Pools, skipping...
03:00:57.111 [14924.14744] <2> vltrun@AddSelectedImagesEjectMedia^655: Media=Z00020 NOT in Eject Volume Pools, skipping...
 
above are the tapes that it identified, but saying that the are in Eject volume Pool in your case, they are not in "Duplicate" Volume pool.
 
your duplicate jobs are picking tapes from some other volume pool. 
you need to add the other volume pool in the eject tab or you need to correct the duplicate tab in vault profile to pick the tapes from Duplicate volume pool.
 
 
<<Edit>>
vltrun@AddSelectedImagesEjectMedia^655: MDA=V10100 selected but not in RBT_GRP=000_00000_ACS, not ejecting
03:00:57.111 [14924.14744] <8> vltrun@AddSelectedImagesEjectMedia^655: MDA=V10102 selected but not in RBT_GRP=000_00000_ACS, not ejecting
03:00:57.111 [14924.14744] <2> vltrun@AddSelectedImagesEjectMedia^655: Media=V10105 NOT in Eject Volume Pools, skipping...
03:00:57.111 [14924.14744] <8> vltrun@AddSelectedImagesEjectMedia^655: MDA=V10122 selected but not in RBT_GRP=000_00000_ACS, not ejecting
 
and also in Robot group 000_00000_ACS, this is the correct one.
 
<<Edit done>>
 
hope this helps.
 
BR549's picture

It is a VTL environment as stated previously. The day-to-day backup jobs are utilizing the VTL Z Series tapes, which predominately reside in the SCRATCH Pool. For OFFSITE Tapes I use the Physical Tape Libray and the V Series tapes.

Therefore the Duplication jobs would be copying from the VTL Z Series tapes to the SL3000 V Series tapes as the physical tapes are the one to be offsited. The SL3000 V Series tapes predominately reside within the Duplicate pool.

This is why there are two pools being utilized on the system. As I stated previously this configuration was working just fine until this past weekend and no changes were made to the Veritas environment before this issue occurred.

But your question made me think about the possible need to add the Duplicate Volume pool to the Eject media from additional (non NetBackup) pools section of the policy. I will give it a try and let you know if that resolves the issue.

BR549's picture

I tried adding the Duplicate pool to the Eject media from additional (non NetBackup) pools section of the policy but it did not resolve the issue.

BR549's picture

Robot group 000_00000_ACS is indeed the correct robot that is to be utilized for the Duplication and ejection of the offsite tapes.

BR549's picture

The output of the vmquery command follows:

Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation. All rights reserved.

C:\>vmquery -m V10102
================================================================================

media ID: V10102
media type: 1/2" cartridge tape (6)
barcode: V10102
media description: ------
volume pool: Duplicate (5)
robot type: ACS - Automated Cartridge System (1)
robot number: 0
robot slot: 0
robot control host: NONE
volume group: duplicate
vault name: ---
vault sent date: ---
vault return date: ---
vault slot: ---
vault session id: ---
vault container id: -
created: 5/10/2012 8:09:05 AM
assigned: 2/2/2013 2:00:51 AM
last mounted: 2/5/2013 2:01:49 AM
first mount: 5/25/2012 1:50:30 AM
expiration date: ---
number of mounts: 137
max mounts allowed: ---
status: 0x0
================================================================================

C:\>

The Robot shows as ACS(1) and not ACS(0)...not sure how that happened...

ScreenHunter_12 Feb. 05 13.17.gif
BR549's picture

I have attached screenshots that show the ACS Robotic configuration.

ScreenHunter_13 Feb. 05 13.22.gif ScreenHunter_14 Feb. 05 13.23.gif
BR549's picture

How would I correct the mislabeled Robot as identified by the vmquery?

watsons's picture

Seem like you have another problem to solve first.. why ACS(0) has become ACS(1)?

If possible, resolve that so that vault can use the correct library ACS(0). Could be the cause as vault could not find anything in ACS(0) which is now ACS(1).

In your sessionID 654, there is this message:

16:06:14.308 [14332.18280] <4> vltrun@cVolInvMgr::LoadInvByRobotNum^654: read 87 volumes for RbtNum=0
16:06:14.308 [14332.18280] <2> vltrun@VaultRobot::pruneMediaBasedOnDbInfo^654: ROBOT=ACS(0) contains:
16:06:14.308 [14332.18280] <2> vltrun@VaultRobot::pruneMediaBasedOnDbInfo^654: Found 0 media in the volume database to eject

It didn't find any media to eject. For deferred eject, we always look at this file  \netbackup\vault\sessions\vlteject.mstr  to see if there is anything wrong.

More info about this file: http://www.symantec.com/docs/TECH35377

RamNagalla's picture

hi,

your robot number looks fine, there is no issue with the robot number. its seen as robot number 0 which is suppose to be.

media ID: V10102
media type: 1/2" cartridge tape (6)
barcode: V10102
media description: ------
volume pool: Duplicate (5)
robot type: ACS - Automated Cartridge System (1)
robot number: 0
robot slot: 0
robot control host: NONE
volume group: duplicate
vault name: ---

but see the volume group, its showing as "duplicate" but Vault is looking in the Group 000_00000_ACS:

you have Volume grup "Duplicate" , move the all volumes  in Duplicate group  to 000_00000_ACS

then try the Vault run.

hope this helps.

SOLUTION
BR549's picture

The only Volume Groups on my Veritas Master Server are duplicate, 000_00001_TLD, My_offsite_volumes and ---. The Volume Pools are None, NetBackup, DataStore, CatalogBackup, Scratch and Duplicate. I have attached screenshots of before and after the changes to this reply.

I had to recreate the 000_00000_ACS Volume Group and assigned the physical tapes to that group. Not sure why that group disappeared. After I created the group and moved the physical tapes to that group I reran the Vault Eject job. The Vault Eject job ran successfully and Tapes were ejected as expected.

I believe this action resolves this issue. I appreciate the assistance in resolving the issue.

ScreenHunter_17 Feb. 06 05.58.gif ScreenHunter_18 Feb. 06 06.00.gif ScreenHunter_19 Feb. 06 06.15.gif
RamNagalla's picture

wow, thats a good news, 

disappering the 000_00000_ACS is the cause of the issue, it does not happen by its own, unless some one did that manully.