Video Screencast Help
Symantec to Separate Into Two Focused, Industry-Leading Technology Companies. Learn more.

Large Job Spanning Multiple Drives

Created: 16 Jul 2014 • Updated: 31 Aug 2014 | 7 comments
This issue has been solved. See solution.

Hi,

 

We have an Quantum i500 Tape Libraray with 6 drives and lots of available slots.  We have a backup job that we would like to create to backup approximately 150TB on a single volume.  Volume is E:\*.*

Subfolders within the root level of the E drive range from 80TB to 10TB in size.  We are exploring NetBackup as a possible solution, but wanted to know if there is a way to setup the job to backup the E:\*.* Folder across multiple tape drives at the same time vs. just a single drive.  We're also concerned about splitting up the job. 

Job1 Backs up Folder 1

Job2 Backs up folder 2

Job3 Backs up Folder 3 & 4

If the users who use this folder create a Folder5, we won't know it is there and won't back it up.  Again, this is my assumpiton, I welcome all feedback and suggestions as we are evaluating NetBackup at the moment.

 

Example for Illsustration Purposes:

E:\Folder1 = 80TB

E:\Folder2= 30TB

E:\Folder3=10TB

E:\Folder4=30TB

Operating Systems:

Comments 7 CommentsJump to latest comment

RamNagalla's picture

hello,

what about the number of files.. are they in small size and large number of files.. or normal files..

i guess you need to look for the  below options

1) flash backup

2) client side dedup with accalator , but if you have the tape dirves it is not supported...

3) SAN media server(make the client as media server)

splitting into multiple jobs will help you to take different tape drives. but i wonder how many days its going to take to compleate the backups.

SymTerry's picture

Hello, 

NetBackup has a feature called Multistreaming. Multistreaming sends data from a single client to multiple tape drives. This assumes that you have a high end client that can send data faster then the tape drives can write to improve efficiency.

Check out TECH31512: What is Multistreaming and how is it turned on? Also see TECH10085: What is the difference between multiplexing and multistreaming?

SOLUTION
Riaan.Badenhorst's picture

Hello,

 

Do you seriously have a 150TB lun assigned to a windows server and its all accessed via you E-Drive?

 

What is kept on this drive, in terms of file type/application?

Regards,

Riaan Badenhorst

You need an OpenVision to see the truth about Backups. Restores are a plus. But that's just Semantics ;)

ITs easy :)

RonCaplinger's picture
  1. Create one policy with Backup Selection=ALL_LOCAL_DRIVES
  2. Create one policy with Backup Selection=E:\Folder1,
  3. Create one policy with Backup Selection=E:\Folder2, etc.
  4. In client properties, EXCLUDE each of the folders (E:\Folder1, E:\Folder2, etc.) that will be backed up individually, for "<<All Policies>>" and "<<All Schedules>>".
  5. In "Exceptions to Exclude List:", add one exception for each of the above folders, for the specific policies you created in steps 2, 3, etc.

How this works:

First, the ALL_LOCAL_DRIVES policy will backup everything, including the entire E: drive,

BUT, because you entered exclusions for the known folders, they will not be backed up in the ALL_LOCAL_DRIVES policy,

BUT, the individual folders *will* be backed up in their own policies, and you can schedule the full backups to occur on different days/weeks, so you aren't backing up the entire 150TB all at once.

AND, if a new E:\Folder* gets created and they don't tell you about it, it will still get backed up by the ALL_LOCAL_DRIVES policy.

Yogesh.Jadhav's picture

Further to Ron's suggestions - Enable multi-streaming so that each folder backup will get a dedicated drive and Backup will take considerable less time compared to a single stream backup.

Thanks & Regards

Yogesh Jadhav
Marianne's picture

You can have E:\Folder* in Backup Selection with 'Allow multiple data streams' in the policy attributes.

This will create a new job for each folder and will therefore not matter if Folder5 is created.

Ensure that Master Server Global Properties has 'Max Jobs per Client' to a value of greater than 1 (4 is a good average).

If you have MPX=1 in Schedule, each folder will go to a different drive.
If you do not see backup speeds that are close to drive capabilities, you may want to multiplex more than one stream to a single drive:
Ensure that the Storage Unit Properties is set to MPX > 1 (4 is good average) and MPX in schedule > 1.
 

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

LilmanHSU's picture

Haha.. everyone seems to be geuninely surprised we have a volume this large on a windows server.  Allow me to elaborate and perhaps this will help sort out the maddess to our reasoning.

This LUN is a 6gb Fiber connection to a Quantum StorNext File System which contains probably around a million or more files including video, audio, pictures and all sorts of other media.  

The client can be whatever we build it to be.  But this multistreaming feature is intersting.. I'll have to research that more.

thanks everyone!