Video Screencast Help

What determines disk image expiration

Created: 04 Sep 2013 • Updated: 10 Sep 2013 | 15 comments
This issue has been solved. See solution.

I just installed a new master server that has 40TB of disk space. I noticed that after the fulls ran this past weekend the previous fulls performed were removed. What determines what is left on disk? I assumed that the high/low watermark did, but it seems that once a new full is performed it expires the previous full. Is that how it works, or is there something that can be changed to retain the backups to disk longer?

Operating Systems:

Comments 15 CommentsJump to latest comment

Kevin Good's picture

There are a number of variables that need to be considered:

Is this Basic Disk, Advanced Disk, DSSU, MSDP?

 

The processes involved with disk utilization are the Image Cleanup, and with MSDP the queue processing and garbage collection...

 

 

Backups are IT-101 (Do Backups) Doing Backups well is an art form. Nobody cares about getting their data backed up... They only care that you can restore it for them!

mph999's picture

The only thing that dictates when an image is eligible to be deleted is the expiration time, which is calculated by the time of the backup and the retention period set. This is set in the scheule, found in the policy.

If using SLP, then the retention period is set in the SLP itself.

The retention periods are in fact a number (known as the retention level) 1, 2, 3, 4 etc ... with each set to a desired period, 1 hr, 1 day, 3 weeks ... whatever you like as they are configerable (apart from retention level 9 which is always 'infinity'). However, when choosing the retention period, your list is given as the time period (eg 1 day) - the retention levels (the numbers) are kinda 'under the covers'. They can be viewed in the Host properites of the master server or by running the retlevel command. However, for this post do not concern yourself with the finer details.

So in your case, I suspect the full backup has a retention period of one week, so when the next cleanup jobs runs it expires the previous backup. This makes it look like the job expires because of the next full running, this is not the case.

I mentioned 'eligible for expiration' - an image doesn't actually get deleted the moment it expires, only when a cleanup job runs which looks for images 'elegible to be expired'. The HWM used with basic disk staging simply triggers a cleanup when the disk is xx% full, but the only images that would be expired, are any that have exceeded their expiration time.

Martin

 

Regards,  Martin
 
Setting Logs in NetBackup:
http://www.symantec.com/docs/TECH75805
 
Marianne's picture

You need to tell us how this 40TB of disk space was configured in NBU. 

High/Low watermarks plays a role if DSSU is configured, else retention levels specified in SLP or schedule.

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

EUAdmin's picture

Sorry for the late response and thanks for the replies.

This Storage unit is setup as a Basic Disk

Mark_Solutions's picture

So basic disk, if you use staging, is capacity managed and here your high / low watermarks come into play

However, if images expire they will be removed

So if the retention of your full backups is 1 week they will be removed after 1 week whether the disk is full or not

Hope this helps

Authorised Symantec Consultant

Don't forget to "Mark as Solution" if someones advice has solved your issue - and please bring back the Thumbs Up!!.

SOLUTION
EUAdmin's picture

The retention of my full backups are for 1 month.

Mark_Solutions's picture

So how large is a full backup of everything?

I knwo you have 40TB of disk but not how large your full backup is - if it is more than 20TB then you could expect it to clear all of the old ones down

What version of NetBackup are you running? There was a bug that constantly cleared down the disk images as soon as they had been duplicated - it was fixed in the latest releases of each version or was temporarily cured by a media server reboot

Authorised Symantec Consultant

Don't forget to "Mark as Solution" if someones advice has solved your issue - and please bring back the Thumbs Up!!.

Marianne's picture

Basic disk with staging to tape?
Or no staging to tape?

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

EUAdmin's picture

Basic disk with staging to tape.

Also, Mark you ask how large my backups are. They are only about 8TB for a full weeks worth of backups including the weekend fulls and the incrementals through the week.

Marianne's picture

Disk staging expiration/cleanup is controlled by retention period and by H/W mark.

What is HWM set to on the Storage Unit?

Enable bpdm log on the media server to see when cleanup is taking place.

This TN explains how cleanup is done since NBU 6.5: http://www.symantec.com/docs/TECH66149

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

Marianne's picture

Have you had a look at the TN in my previous post?

Do you have bpdm log enabled on media server(s)?

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

Mark_Solutions's picture

As it stands with what you have said you would expect to have at least 2 or 3 Full backups retained on disk based on its size and retention period

If you do not have any retained then something is clearly wrong.

What version and patch level of NetBackup are you using? - As i said there were bugs that caused it to clear everything down - though if that were the case i would expect the disk to be totally empty once it had all been duplicated to tape.

So that we can get a better idea could you run the following commands and attach the outputs (feel free to change server names in the output) - all from \netbackup\bin\admincmd\:

bppllist -allpolicies -U>c:\policies.txt

bpstulist -g -U>c:\stus.txt

bpschedule -U>c:\stusched.txt

Thanks

Authorised Symantec Consultant

Don't forget to "Mark as Solution" if someones advice has solved your issue - and please bring back the Thumbs Up!!.

EUAdmin's picture

I must apologize. I may have wasted everyone’s time. It looks a though it is retaining the backups as expected. I don't know if I was overlooking it when I was doing the search in the catalog or what, but I have the fulls for this past weekend and last weekend. Thanks for everyone’s insight though.

Mark_Solutions's picture

If you did experience the bug that I mentioned earlier then you may have been right!

Do make sure that you are patched right up to date just in case

Authorised Symantec Consultant

Don't forget to "Mark as Solution" if someones advice has solved your issue - and please bring back the Thumbs Up!!.