Performance improvements for tape in NetBackup 7.1

Article:TECH143890  |  Created: 2010-11-10  |  Updated: 2011-08-24  |  Article URL
Article Type
Technical Solution


Asynchronous form of the write filemark command

Data is written to tape in a series of fragments, which are separated with a logical element called a FileMark. NetBackup has a specific format for the fragments it writes, consisting of small blocks of data to represent the source of the following data (a list of backup ids). Following the list of backup ids is a much larger set of data blocks that contain the backup data itself. A single backup will consist of at least one fragment, and may be made up of thousands of fragments. As tape drive technology has advanced, the time required to write data has significantly decreased, while the time required to write a filemark has remained about the same. For customers writing many small images, such as database transaction logs, this means that the maintenance tasks involved with writing a NetBackup image to tape now take a substantial amount of the time required to write the image.

When a tape drive receives a request to write a standard (synchronous) FileMark, it must first flush all buffered write data to tape before writing the file mark and reporting success, which can take 1 to 2 seconds to perform. It is possible to use a flag for the write filemark command to the tape drive, called the Immediate flag, that allows the write filemark operation to be buffered (asynchronous) on the drive, in which case the operation only takes ~ 3 ms, and does not require the drive to flush buffered data to tape.

NetBackup 7.1 will still use the synchronous form of the write filemark command for checkpoints, and when a client has finished sending data for an image. NetBackup will use the asynchronous form of the command when there is still more data to be written for an image, such as TIR, maximum fragment size, next image start, and single image failure in an MPX group.

Empty Header

Another thing that NetBackup does to help keep track of the location of data on a tape is it writes what is called an empty header (EH) after the end of every group of images written to a tape. During Multiplexed backup (MPX), as the final backup completes, an empty header is written to the media. The assumption was that no more backups would be added to this job, but that is not really true. If there are other jobs started then the tape is backed up, the EH is over-written, and NetBackup continues to write data to the tape. Modern tape drives are sensitive to backing up, and this can result in delays of up to 20 seconds while the tape is readied to receive more data.

NetBackup 7.1 will now check for new jobs prior to writing the EH.

MPX Wait Delay

When there are multiple images being written to the same fragment on tape (referred to as MPX), NetBackup prepares the tape, then waits 5/10 seconds for other jobs that may also be able to write to the same fragment. NetBackup checks for new jobs once per second. If bptm has already written the MPX headers for the images in that fragment, then they will need to be over-written, with similar delays to that for over-writing an empty header.

NetBackup 7.1 will check for new jobs or for available data before starting to write an image fragment. If there is data available to be written to tape then bptm will start writing and ignore any new jobs for 4 seconds. This will reduce the possibility that these small header files written to tape will need to be over-written. Additionally, NetBackup will now check for new jobs 10 times per second, instead of once per second, which should decrease average response time from .5 seconds to .05 seconds.

Write-side chaining

NetBackup 6.x uses write-side chaining, which allows non-MPX images from multiple sources duplicating to the same storage unit to use the same tape without the need to reassign the resources between each job.

NetBackup 7.1 introduces read-side chaining, which is essentially the same, for images on disk storage units. Read-side chaining will only work if write-side chaining is also being used.

Deferred Image Validation for Duplications

If a series of small images are going to be duplicated, the penalty for writing a filemark can be eliminated if the images are grouped into batches, with verification of write integrity delayed for a certain number of images. If one of the images fails to duplicate then the entire batch would fail, but as this is a copy of the data, rather than the original data, the performance benefits outweigh the "risk" of having to rerun a duplication batch.

create /usr/openv/netbackup/db/config/DEFERRED_IMAGE_LIMIT with value from 1 - 128 (default 8) on the master server
create /usr/openv/netbackup/db/config/DEFERRED_IMAGE_KBSIZE with desired max image size to be deferred (default 1G) on the master server

Look for "read_config_file" messages in bptm or admin log to verify controls.

Append-Only Mode

SCSI Stream Commands (SSC-4) has introduced a new data protection feature for tape drives called "Append Only Mode".

  • This feature causes the drive to prevent the overwrite of data on a cartridge unless the writing application performs a pre-authorization for the overwrite.
  • Pre-authorization for overwrite is a done with a new SCSI command (Allow Overwrite) that informs the drive that the data block at the current location is going to be overwritten.
  • Any attempt to overwrite a data block without pre-authorization returns a write protect error.
  • IBM LTO-5 drives support this feature (even with LTO-4 media).
  • Once set, LTO-5 drives remain in append mode until power cycled.


NetBackup 7.1 contains many performance improvements for customers using tape drives.

Article URL

Terms of use for this information are found in Legal Notices