Video Screencast Help
Symantec to Separate Into Two Focused, Industry-Leading Technology Companies. Learn more.
Backup and Recovery Community Blog

One NetBackup Master/Media server, one tape drive, and millions of files

How do you back up millions of files when you only have a NetBackup Standard client and a single tape drive?
Created: 09 Jun 2014
John Meaders's picture
0 0 Votes
Login to vote

Earlier this year I was engaged to install NetBackup on a Solaris server which previously used a competing backup product.  This server only backs itself up and is attached to a single drive tape autoloader.  I initially thought I would use a single NetBackup policy to back this server up but it quickly became apparent this wouldn’t work as one filesystem took forever to back up - which would push the backup way beyond the desired 8 hour backup window.

This filesystem contained millions of small files.  At this point, you are probably thinking - simple - just use FlashBackup.  Unfortunately, the customer was only licensed for the Standard client and this project was already an out-of-budget expenditure, so further expenditures would be very difficult.  The other items on this server that needed to be backed up were Oracle dump files, along with the OS directories.

I performed some research and after viewing a couple threads here on Symantec Connect I felt I had found a solution.  I would multi-stream the backups with 4 streams allowed on the client, and since I only had one tape drive to work with, I would use multiplexing.  The customer was agreeable to this approach.

My solution involved changing the Maximum jobs per client to 4 in Global Attributes, and using 3 Standard policies to back the server up along with the Catalog Backup policy.  The first policy (A) backs up the filesystem with millions of files:  allows multiple data streams, has a priority of 20, and the schedules have a maximum multiplexing of 4.  The second policy (B) backs up the OS files:  allows multiple data streams, has a priority of 10, the schedules have a maximum multiplexing of 4, and it uses ALL_LOCAL_DRIVES.  The third policy (C) backs up the database dump directories:  allows multiple data streams, has a priority of 0, and the schedules have a maximum multiplexing of 4.  The OS policy has an exclude file that excludes the database dump directories and the filesystem with millions of files.  The database policy has an exclude file that excludes the directories containing the active database files and archive logs.

It is important to note that for all of these policies to multiplex on the same tape and drive, they all need to use the same multiplexing.  In this case, I used 4.  The storage unit that I used also has multiplexing set to 4.

All of the policies are scheduled with the same backup windows.  What occurs is policy A starts first (as it has the highest priority) and gets one stream (there is only one file system), then policy B gets two streams (since it has the next highest priority), and policy C gets one stream.  Once policy B completes, policy C runs with three steams.  Policy A tends to run for the entire length of the backup.  A full backup takes about 6 hours.  With the previous solution a full backup took about 8 hours.

I recently re-visited the customer to see how things are going.  They are very happy with the solution and it is still running great.

See what a single tape drive and a single master/media server can accomplish with a little creative policy and schedule creation?