Video Screencast Help

Maximize Tape Cartridge Usage

Created: 12 Dec 2012 • Updated: 17 Feb 2013 | 12 comments
This issue has been solved. See solution.

We recently installed LTO5 tape drives in our environment.  We have noticed since this change that many of our LTO5 tapes are going offsite with very little written to them.  This is causing us to go through many more tapes then we should be.  I would like to find out any Netbackup recommendations for writing to backup tapes in a way that provides maximum utilization of the tape. 

Comments 12 CommentsJump to latest comment

Marianne's picture

Without knowing anything about your environment, here are a couple of things to look at:

  1. Media sharing 
    By default media is assigned to one media server.
    You can enable media sharing globally or create media sharing groups:
    See these topics in NBU Admin Guide I:
    “Configuring unrestricted media sharing” 
    "Media media sharing with server groups"
  2. Maximum number of partially full media per volume pool
    Details in Admin Guide I 
    Jobs may queue with setting instead of using a new tape
  3. Limit number of volume pools
    Self-explanitory - the more pools, the more media will be used.
  4. Limit retention levels
    NBU does not mix retention levels by default.
    This can be enabled, but is not recommended - it decreases media usage in the short term, but creates major headaches in the long run as tapes cannot be reused when short retentions expire.

Hope this helps!

 

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

Nicolai's picture

I would absolute look at "Maximum number of partially full media per volume pool" as Marianne suggest. Likely you are removing tapes from the robot faster than Netbackup can fill the tapes because of the increased tape capacity. What before took 2-3 tapes now fit in a single cartridge.

Assumption is the mother of all mess ups.

If this post answered your'e qustion -  Please mark as a soloution.

Andy Welburn's picture

It could also depend upon how your policies, schedules, storage units etc are set up & how big your environment is with regards to number of clients, drives, policies etc etc- which in turn would affect how many tape drives & hence tapes are utilised at the same time.

e.g. all things being equal, due to policy/schedule/STU limits, you *could* have one job going to each of 5 or more drives concurrently but it is possible that your system(s) could perform just as well with all those jobs going to just one tape drive. This could have worked no problem for the lower capacity LTO3 media but obviously has its 'media utilisation shortcomings' when using LTO5 media.

This is purely hypothetical as we know nothing of your environment! wink

bills's picture

We struggle with the same issue in my environment.  What we try to do is, because we know almost exactly how many tapes our different data sets will fit on, we allocate that many tapes per pool, and do away with scratch pools.  For example, we know that the daily offsites fit on 10 tapes, the weekly fulls fit on 64 tapes, etc.  Each of these sets writes to a different pool, so every day/week we assign the needed tapes.  Some of this is scripted, but it still requires a lot of time babysitting tapes, and is hardly what I would call an optimal solution - but, we usually only send off a few non-full tapes.

We've thought of using the max partially full setting, but the problem there is that it becomes a bottleneck.  We have 18 drives - if we want to utilize all of them, we'd need the max partially full number to be 18, with the risk of sending off 18+ partially full tapes.  If we say the max partially full tapes is 5, then we're limiting ourselves to only 5 drives at a time, and will likely exceen our backup window.

I'd be very interested if anyone has a more elegant solution!

Bill

Andy Welburn's picture

No matter which way you go that looks like a balancing act

i.e. either:

  • use as many drives as possible at the risk of having more partially full media, OR
  • restrict the number of partially full media & run the risk of exceeding your backup window by not utlising available drives

You could try & play around with it a little, altho' it has the potential to carry a little risk - start afresh one day/week as far as media is concerned (all 'unused') & see how many media it uses in 'one sitting' & how close to the backup window the saves finish. You could then look at setting the mpf value at a level such that the backups still complete in the window.

Never actually used any mpf setting & *always* use scratch so I'm not 'babysitting'. Tended to try & limit the actual number of media utilised by restricting via jobs per policy & MPX settings etc - but then we only ever had 4 drives to consider!

bills's picture

Agreed - I've always used scratch pools and not worried about it.  At my last job we only sent tapes offsite when they were full - but there were a few cases when tapes would sit in the robot for months instead of going offsite.  In fact NetBackup tends to contribute to this by the way it selects tapes.  When faced with multiple choices in the robot, it will select the most recently written to.

In my previous experience, we had a pool whose jobs frequently spanned tapes.  So jobs would run, and leave partially full media - then more jobs would run the next cycle, would start on some of these tapes, span to new tapes (since NBU will choose empty tapes to continue spanned images if it can), then leave those tapes partially full.  The next cycle, it would choose the partial tapes from the 2nd cycle, since they were the most recently written to, effectively orphaning the tapes from the first cycle...

Bill

Andy Welburn's picture

When faced with multiple choices in the robot, it will select the most recently written to.

Really? My experience was always seemed to be the opposite, which meant that if ever I wanted to get to the minimum number of partially full media in the library at any one time I had to remove same from the library & 'start again' as it were.

But your next statement does ring true to what we experienced!

 

since NBU will choose empty tapes to continue spanned images if it can

Oh yes - there's logic behind it but that found 'annoying' too.

bills's picture

Andy:

I spent quite a bit of time a few years ago trying to figure out why I was being left with orphaned tapes in the robot - even opened up a case on it - including a lengthy discussion with you (https://www-secure.symantec.com/connect/forums/how-does-netbackup-choose-which-active-tape-use).

As I recall, talking with the support person that had talked with the engineers, there was some logical reason (that I disagreed with) that said the most recently used tape would be used, IF there were multiple available unmounted tapes matching retention, density, etc.

I no longer have my testing results, but I was able to verify this reliably.

 

Bill

Netbackup Admin Guy's picture

Thanks for all the feedback so far.  Here is a little information regarding our configuration. 

 

  • OS: Windowws 2008 R2 SP1
  • Netbackup 7.5.03
  • 1 master server and 3 media servers (one of the media servers is the master)
  • 1 robot
  • 16 - LTO5 tape drives and cartrdiges
  • All backups have 2 copies.  Copy 2 goes offsite.
  • Enable unrestricted media sharing for all media servers is turned on
  • Allow backups to span disk is turned on
  • All backups to span tape media is turned on

All of our backups are configured to create two copies.  Copy 1 always uses volume pool BCW.  Copy 2 is configured with a different volume pool that depends on the type of backup.  Such as: dup_daily, dup_weekly, dup_monthend, dup_yearly.  Copy 2 goes offsite.

I inherited this configuration but have always questioned it.  Do you see any reason why so many voloume pools are required?  

Any other feedback would be appreciated as well.  Thank you so much!

 

 

mph999's picture

Number of volume pools is up to you, the less there are the less chance there is of using more media.

If the volume pools you list all have the same retention, then you could make do with one.  If they have different retentions, than it makes no difference, as different retentions will goto different media by default (and should not be changed).

The fewer different retentions you have, the less media you will use

(Don't mix retentions though, as this could tie all your media up with the longest retention you have which could be mean you completly run out ...)

The most effective setting to use less media is to reduce the max number of partially full media (per volume pool), but this could lead to queued jobs, as I think Nicolai mentioned.

Martin

 

 

Regards,  Martin
 
Setting Logs in NetBackup:
http://www.symantec.com/docs/TECH75805
 
Andy Welburn's picture

If the volume pools you list all have the same retention, then you could make do with one.  If they have different retentions, than it makes no difference, as different retentions will goto different media by default (and should not be changed).

The fewer different retentions you have, the less media you will use

(Don't mix retentions though, as this could tie all your media up with the longest retention you have which could be mean you completly run out ...)

^ This.

Let NetBackup manage your tapes as best it can. It is nice to have everything 'compartmentalised' but we have seen it taken to extremes in a lot of cases on the forum.

Marianne's picture

All of our backups are configured to create two copies. .... Copy 2 goes offsite.

Can we assume that you are using Inline Tape Copy?
And that you don't really have a problem with onsite media (as they can still be appended to on a daily basis) but with the tapes going offsite?

If this is the case, it might be worth your while to run duplications afterwards - using something like Vault or SLPs. I know that with Vault you can limit the number of read drives and altreadhost. This will ensure more efficient use of media used for duplication.

Depending on the amount of data to be duplicated, this may be time consuming, but you have to weigh up media savings vs the convenience of having all copies completed by the time the backup is done...

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

SOLUTION