Virtual Tape Libraries, Tape Capacity and Media Management
NetBackup does not treat media and the contents of data (images) written to Virtual Tape Libraries (VTLs) any differently to real Tape Libraries.
A mile high view of the process after the 'expiry date' of each NetBackup image has passed :
- Removes the image from the image database
- Decrements the number of valid images on the media in question in the media database (a table within the EMM db since NetBackup 6).
- Once the number of valid images in the media db reaches zero, the 'assigned' field in the volume database (a table within the EMM db since NetBackup 6) is cleared
Optionally the tape is moved back into the 'Scratch' pool (if configured).
Note: It is best practice is to configure a Scratch pool.
Nothing in the above process touches either the media or library. For "real" libraries, the media can be out of the library and offsite (again this is best practice), and the above cleanup process completes successfully.
As VTLs use disk space to store the data, VTLs have a finite capacity. Often VTLs are configured where the amount of disk space is over-committed for the virtual media. This can result in the VTL filling up and unable to accept more data.
As NetBackup expires images, the now available (from NetBackup perspective) media still consumes valuable disk space on the VTL.
Note: Over-commit disk space is a simple calculation of the number of virtual media times the size of each media exceeding the total disk capacity of the VTL.
NetBackup may select new (as yet unwritten) media for new backups (instead of the previously expired media) resulting in failed backups due to the over-commit of VTL disk space.
Write a small amount of data to the beginning of each newly 'unassigned' media.
The VTL now 'knows' the media only contains x KB of data instead of the previous GBytes/TBytes of data. The VTL will then be able to reclaim this space for other media.
NetBackup best practice to 'write a few KB of data' is to use the 'bplabel' command and relabel the media AFTER the media has expired. Care needs to be taken with the bplabel as it does not check if media has valid data (incorrect use of bplabel can overwrite valid media resulting in the inability to restore data - you have been warned).
NetBackup hook to simplify this process is the 'media_deassign_notify' script.
- Promote the media_deassign_notify from the goodies directory (i.e. move from netbackup/bin/goodies/media_deassign_notify to netbackup/bin)
- Modify media_deassign_notify script to call 'bplabel' command with appropriate options to suit your environment/setup.
ONLY label media in the 'Scratch' pool.
For the protection of your existing backups and your sanity, configuring the 'Scratch' pool is highly recommended for this process.
Why: Media is moved from the Scratch pool once assigned (i.e. media now contains valid image).
The bplabel command will fail (because the media is now in a different pool).
This protects against the race condition of other backup jobs attempt to utilise the newly released media
within the window of 'media deassigned' and your bplabel command being called and executed.
Parameters passed to media_deassign_notify are:
"-----------------------------------------" " MEDIA ID: $1" "LEGACY MEDIA TYPE: $2" " BARCODE: $3" " ROBOT NUMBER: $4" " ROBOT TYPE: $5" "----------------------------------------"
The modifications to the 'media_deassign_notify' script should also check the robot number and only re-label virtual media.
Identify 'scratch' pool:
/usr/openv/volmgr/bin/vmpool -list_scratch Scratch Pools ============= Scratch
Where 'Scratch' is the name of the scratch pool.
Identify 'robot number'
# /usr/openv/volmgr/bin/tpconfig -d Id DriveName Type Residence Drive Path Status **************************************************************************** 0 STK.T10000B.004 hcart TLD(1) DRIVE=5 /dev/nst4 UP 1 STK.T10000B.005 hcart TLD(1) DRIVE=4 /dev/nst3 UP 2 STK.T10000B.006 hcart TLD(1) DRIVE=3 /dev/nst2 UP 3 STK.T10000B.007 hcart TLD(1) DRIVE=2 /dev/nst1 UP 4 STK.T10000B.008 hcart TLD(1) DRIVE=1 /dev/nst0 UP Currently defined robotics are: TLD(1) robotic path = /dev/sg8
Robot number : '1' - Note the "TLD(1)" reference to above output.
Query media details (identify which pool, if assigned and media type)
vmquery -m SYM003 ================================================================================ media ID: SYM003 media type: 1/2" cartridge tape (6) barcode: SYM003TA media description: --- volume pool: Scratch (6) robot type: TLD - Tape Library DLT (8) robot number: 1 robot slot: 3 robot control host: mhmedia volume group: 000_00001_TLD vault name: --- vault sent date: --- vault return date: --- vault slot: --- vault session id: --- vault container id: - created: Fri 24 Jun 2011 01:15:21 PM EST assigned: --- last mounted: Tue 18 Oct 2011 11:48:04 AM EST first mount: Fri 24 Jun 2011 01:30:25 PM EST expiration date: --- number of mounts: 3 max mounts allowed: --- ================================================================================
The above media is 'deassigned' - Note the 'assigned:' field is blank. Also assigned media will contain an extra field of "status: 0x0" as the last entry.
# bplabel -m SYM003 -d hcart -p Scratch Media is already NetBackup format, media id = SYM003, overwrite it y/n (n)? y
Note: Add the '-o' switch to automatically answer 'y' to any questions.