Video Screencast Help
Symantec to Separate Into Two Focused, Industry-Leading Technology Companies. Learn more.

Netbackup Storage LifeCylce Policies and DD Pool filling up

Created: 29 Apr 2013 • Updated: 09 Jun 2013 | 21 comments
This issue has been solved. See solution.

Hi all,

Recently we implemented Deduplication (and soon will be replication to another server), using Disk Pools and Storage LifeCycle Policies.

Before we had just Basic Disks for Storage Units for Online backups to disk.

Since moving to SLP we are constantly getting error 129 (Storage unit is full). We never had this issued with a Basic Disk Setup. It just worked as it should and did what the High and Low watermarks are supposed to do. With SLP these watermarks don't work at all. The watermarks are the default of 98% and 80% (high and low respectively).

Why does it not work ? Why does it not perform in the same manner ? Why is it not consistant ?

How do I fix this issue so that the server just expires images (the oldest ones) from the online disk storage when it needs the space to perform newer ones like it did before ?

Thanks

Operating Systems:

Comments 21 CommentsJump to latest comment

Marianne's picture

SLPs work different - the retention levels that you specify in the Backup and Duplication destinations are applied to images.

You also need to confirm that duplications are actually going through - until duplications have completed successfully, the backup copy is assigned Infinity retention. After successful duplication is actual retention applied.

Have you read up about Storage Lifecycle Policies in Admin Guide I yet?

There is also this Best Practice Guide: http://symantec.com/docs/HOWTO73205

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

JasonP_76's picture

I am not using an appliance. It is a PureDisk Pool created in NetBackup 7.5

Nicolai's picture

You will need to configure the SLP to perform staging operations.

Go create a new SLP:

Add a Operation -> "Used for Backup". Set "Staged Capacity Manged Desired cache Period" to 2-3 days

Add a new Operation: This time for "Duplication". Retention type must be fixed this time.

The SLP best practices is really good - I recommend it highly : 

http://www.symantec.com/docs/TECH75047

Assumption is the mother of all mess ups.

If this post answered your'e qustion -  Please mark as a soloution.

JasonP_76's picture

Marianne,

So what are the high and low watermarks then supposed to do then on the Disk Pool if SLP's are involved ? Do they even do anything then ?

Being forced down the route to use SLP's just to get duplication to work, and as a result completely change the way backups are retained is crazy. 

So basically you are telling me that if I back anything up using SLP's the data has an infinite retention until it is duplicated somewhere else and even if it is it holds onto its retention period and does not expire sooner even though there is another copy somewhere else ? (hence the reason why the disk is filling up, because it won't expire duplicated copies prior to its actual retention end date)

The way BasicDisk worked was so much easier and less hassle. Just set the policy, point it to a Storage Unit and leave it and Netbackup will just do everything in the background and you never have to worry about it erroring for disk space, because it just worked and complied to the high watermarks.

I have not changed anything the way we back things up from when we had it in a BasicDisk setup (retentions and staging to tape). And even SLP's seem a little archaic that you can't set when you want the Duplication to Tapes (Or any duplication, even cross site) to run like they do when you set a staging with a BasicDisk Storage Unit. They just run when ever they feel like it. There seems to be no flexibility using SLP's, and it seems your hands are tied with using them when you want to set up duplication.

Nicolai's picture

Don't use SLP unless there is a gain :-)

The DSSU is working really good.

SLP does provide flexibility - lots of it - But not all backup configuration need the flexibility of SLP.

Assumption is the mother of all mess ups.

If this post answered your'e qustion -  Please mark as a soloution.

JasonP_76's picture

Nicolai,

I don't see any option for "Staged Capacity Manged Desired cache Period"

Nicolai's picture

GUI -> Storage -> Storage Lifecycle Policies -> Right click -> New Storage Lifecycle Policy ->  Add (here you will find it)

Assumption is the mother of all mess ups.

If this post answered your'e qustion -  Please mark as a soloution.

JasonP_76's picture

Nicolai,

I have to use SLP's to have Duplication of data to another Disk Pool on another site. There does not seem to any other option when I started going through the Netbackup Docs for DeDuplication and Cross Site duplication.

JasonP_76's picture

I know how to get to add an SLP.

I am saying that option you are talking about is not listed in ANY of the option within creating an SLP

Nicolai's picture

Is that pool within the same Netbackup domain ?. if not you need to look at Netbackup AIR

Assumption is the mother of all mess ups.

If this post answered your'e qustion -  Please mark as a soloution.

JasonP_76's picture

The pool will not be in the same domain as because the source media server is also the master server and if the master server goes offline the media server it would replicate to within the same domain would not work without the master, so we have to set it up as two seperate netbackup domain's.

But that is later. I am trying to sort out getting the disk storage from not filling up and Netbackup to manage the disk usage and expire backups as and when is needed to release space for backups to work.

Nicolai's picture

The only approved way to replicate data from one Netbackup domain to another is via Netbackup AIR. The use of MSDP is required.

Assumption is the mother of all mess ups.

If this post answered your'e qustion -  Please mark as a soloution.

JasonP_76's picture

Nicolai,

I understand that. And we will have that in place. We already have a DeDuplication Disk Pool in place and working. Just need help to sort out the disks filling up when using this DP and Netbackup managing the disk usage automatically.

Can you help me get that sorted first ?

Jaykullar's picture

NetBackup will not "manage disk space" unless you configure your SLPs to do this. As Nicoli previously mentioned, when configuring your SLP check the "staged capacity managed" and choose your desired cache period, or expire after duplication.

The other options are to either reduce your retention period or increase your disk space.

Nicolai's picture

I think I can - But I am missing the bigger picture of duplication, replication and stagning. 

Can you please explain what you want to do - Is the problem the deduplication pools fill up?

Assumption is the mother of all mess ups.

If this post answered your'e qustion -  Please mark as a soloution.

JasonP_76's picture

Jaykullar,

As I mentioned when configuring my SLP's there is NO option to check anything for "Staged Capacity Managed" or anything that sounds remotely similar. There is NO ability to choose any cache period. 

RLeon's picture

Let's start again...

From the Nbu Admin guide, SIS storages like PureDisk and, by extension, MSDP, do not support "Capacity Managed" mode in SLP:

Capacity managed is selectable for any disk storage unit that is allowed in a storage lifecycle policy. However, for the disk types that support single-instance storage (SIS), Capacity managed functions to various degrees. In order for Capacity managed to operate, NetBackup must know how much space a backup image uses. With SIS enabled on the storage unit, NetBackup cannot know exactly how much space a particular backup image occupies.
The following storage unit configurations use SIS:
 - PureDisk storage units
 - Some OpenStorage storage units, depending on the characteristics of the OpenStorage partner.

Some people only use the term "SIS" to refer to some variant of this deduplication technology that works at the file level, but it will have to be a discussion for another time...

The Admin guide is just saying - in a long winded way - that the "Capacity Managed" mode will only work on non-deduplicated AdvancedDisk Disk Pools.
BasicDisk is out of the picture because SLPs do not work with them.

When the high water mark setting of a PureDisk disk pool is reached, backups will fail, and Image Cleanup will be triggered to expire images until it falls below the high water mark, i.e., it will not necessarily reach the low water mark. Dedup disk pools do not use the low water mark setting, only capacity managed AdvancedDisk storages do, as the Nbu Dedup guide puts it:

The Low water mark has no affect [sic] on the PureDiskVolume.

You would expect that whenever the Image Cleanup is triggered by the high water mark, images will actually be expired to clear some space, but this is not always the case.
Specifically, images that have not been duplicated to the next storage down the SLP hierarchy will not be expired (regardless of its fixed retention setting), and images that have been duplicated but have not lived through its fixed retention setting will also not be expired.

To solve your problem, in your SLP you can either shorten the retention for the images that will be staged in the PureDisk pool (don't worry, they won't expire until they are duplicated to tape), or you can set the retention as Expire After Copy, which does exactly what it says.
The duplication target/destination retention (tape, in your case) can be set to a longer fixed retention. It does not have to have the same retention as the staging PureDisk storage images.

That is one of the main differences between BasicDisk Staging and using SLP to duplicate from disk to tape. In BasicDisk Staging, images in the staging disk storage and in the target storage have the same retention, as in:

BasicDisk Storage(1 month retention) ------> Tape(1 month retention)

But with SLP, you can have a different retention at every stage:

Dedup Storage(1 week retention) ------> Tape(1 month retention)

...Although with SIS/Deduplication you do lose the ability to have the (staging) disk storage running in capacity managed mode.

SOLUTION
JasonP_76's picture

Rleon,

Thanks for that. Makes a little more sense now. Shame it does not manage the capacity the way it used to when it was only in BasicDisk mode.

I have changed the retentions on the all disk based (staging) backups and left the normal ones for the tapes.

Hopefully this will help it out and reclaim disk space.

RLeon's picture

Thanks Jaykullar.

JasonP_76, please note that changing an SLP's setting would only affect new backup images. For the existing images created before the change, or the ones that are SLP-incomplete (images that have not been duplicated all the way to the last stage), those would still have the old settings/retentions. For those images - depending on how much space you have left to take in new backups - you may have to manually shorten their retention by using the bpexpdate command. Please refer to the Nbu Commands Reference guide for the options.

Also, having expired an image does not mean that the dedup disk space will immediately be reclaimed. This is done internally and automatically according to a set schedule. You can find out more from the Nbu Dedup guide under Deduplication Queue Processing, and
http://www.symantec.com/docs/TECH124914

SLP scheduling is planned for NetBackup 7.6.