Video Screencast Help
Protect Your POS Environment Against Retail Data Breaches. Learn More.

When backup sets are deleted under DLM

Created: 09 Jun 2012 • Updated: 09 Jul 2012 | 9 comments
Language Translations
pkh's picture
+8 8 Votes
Login to vote

In BE 2012, when you use fixed disk, the storage that would be created is called Disk Storage.  When you store a backup set in a Disk Storage, you can specify the retention period for the backup set.  The management of this backup set is done through a mechanism called Data Lifecycle Management (DLM).  DLM is explained here

https://www-secure.symantec.com/connect/blogs/data-lifecycle-management-be-2012

Note that in addition to your specified retention period, DLM will not groom the last copy of the latest recovery point chain and the grooming process takes place every 4 hours or when a low disk space condition is encountered.  This makes it difficult to visualise what is on your disk and visualising what is on your disk is critical when you are trying to squeeze as much as possible onto your disk.

You can turn off the last copy feature of DLM using the procedure in this document

Backup 2012 will not overwrite the last or recent backup set. Unable to delete the last backup set.

This is needed if you want to delete the last backup set of a server that has been de-comissioned.

In this article, I would use some example/scenario to illustrate how DLM works and when backup sets are deleted.

Minimum Retention Period, Only DLM At Work

The retention period for the backup set is 1 hour.  This is so that the backup sets can be deleted as soon as possible and they are only retained because of the other criteria specified by DLM.

Full + Incremental Backups

We have a full backup (Fx) running every Sunday and incremental backups (Incrxx) on Monday to Friday.  All jobs starts at 1 a.m. and finshes at 2 a.m.  The jobs are as follows:-

Week Sun Mon Tue Wed Thu Fri Sat
1 F1 Incr11 Incr12 Incr13 Incr14 Incr15  
2 F2 Incr21 Incr22 Incr23 Incr24 Incr25  
3 F3 Incr31 Incr32 Incr33 Incr34 Incr35  

Even though the retention period for the backup set created by full backup job F1 expires at 3 a.m. on Sunday, it is not groomed even when incremental backup job Incr11 runs.  This is because backup set F1 is the last recovery point of the server.

Likewise, backup set Incr11 is not groomed when job Incr12 runs, even though its protection expires at 3 a.m. on Monday.  This is because Incr11 is the last recovery point of the server.  Backup set F1 is also not groomed because backup set Incr11 depends on it.  So it continues, after job Incr15 has completed, you would have backup sets F1, Incr11, Incr12, Incr13, Incr14 and Incr15 on your disk.

When job F2 runs, all the backup sets from Week 1 are still not groomed because Incr15 is the last recovery point of the server and it depends on the rest of the backup sets.  After job F2 has completed, it is now the last recovery point of the server and it is not dependent on the backup sets from Week 1, so these can be groomed.  For a maximum of 4 hours until the next grooming cycle, you would have backup sets F1, Incr11, Incr12, Incr13, Incr14, Incr15 and F2 on your disk.

In this scenario, your disk must have adequate space to hold 2 full backups plus 1 week of incremental backups.

Full + Differential Backups

We have a full backup (Fx) running every Sunday and differential backups (Diffxx) on Monday to Friday.  All jobs starts at 1 a.m. and finshes at 2 a.m.  The jobs are as follows:-

Week Sun Mon Tue Wed Thu Fri Sat
1 F1 DIff11 DIff12 DIff13 DIff14 DIff15  
2 F2 DIff21 DIff22 DIff23 DIff24 DIff25  
3 F3 DIff31 DIff32 DIff33 DIff34 DIff35  

Like the incremental jobs case discussed earlier, when job Diff12 runs, backup sets F1 and Diff11 are still not groomed even though their overwrite protection has expired.  This is because backup set Diff11 is the last recovery point of the server and it is dependent on backup set F1.

After job Diff12 has completed, backup set Diff12 is the last recovery point of the server and it is only dependent on backup set F1.  It is not dependent on backup set Diff11, so backup set Diff11 can be groomed.  Until the next groom cycle which can be as long as 4 hous away, you would have backup sets F1, Diff11 and Diff12 on your disk.  When job Diff13 runs, you should only have backup sets F1 and Diff12 on your disk.

When job F2 runs, backup sets F1 and Diff15 is still on your disk because Diff15 is the last recovery point of the server and it depends on backup set F1.  After job F2 completes, backup set F2 is now the last recovery point of the server and it is not dependent on either backup sets F1 or Diff15, so they can be groomed.  Until the next grooming cycle, you would have backup sets F1, Diff15 and F2 on your disk.

In this case, the maximum disk space that you need to cater for is 2 full backups and the size of a Friday differential backup.

 

DLM + A 2-Weeks Retention Period

 

Full + Incremental Backups

 

We have a full backup (Fx) running every Sunday and incremental backups (Incrxx) on Monday to Friday.  All jobs starts at 1 a.m. and finshes at 2 a.m.  The jobs are as follows:-

Week Sun Mon Tue Wed Thu Fri Sat
1 F1 Incr11 Incr12 Incr13 Incr14 Incr15  
2 F2 Incr21 Incr22 Incr23 Incr24 Incr25  
3 F3 Incr31 Incr32 Incr33 Incr34 Incr35  

Once job F2 completes, backup set F2 is the last recovery point of the server, the chain of backup sets from Week 1 are no longer required under the DLM rule and can be groomed.  However, these are retained because they are still protected by the their retention periods.  Backup set Incr15 will only expire at 2 a.m. on the Friday of Week 3.  Although the retention period for backup sets F1, Incr11, Incr12, Incr13 and Incr14 has long expired, they are still not groomed because backup set Incr15 depends on them.  Only after the expiry of the retention period for backup set Incr 15, i.e. after 2 a.m. on Friday of Week 3, then the entire backup chain from Week 1 can be groomed.

In this scenario, you need to have enough disk space for 3 full backups and 15 incremental backups.

 

Full + Differential Backups

As before, we have a full backup (Fx) running every Sunday and differential backups (Diffxx) on Monday to Friday.  All jobs starts at 1 a.m. and finshes at 2 a.m.  The jobs are as follows:-

Week Sun Mon Tue Wed Thu Fri Sat
1 F1 DIff11 DIff12 DIff13 DIff14 DIff15  
2 F2 DIff21 DIff22 DIff23 DIff24 DIff25  
3 F3 DIff31 DIff32 DIff33 DIff34 DIff35  

As with the differential backups case discussed earlier, once job F2 completes, backup set F2 is the last recovery point of the server, the chain of backup sets from Week 1 are no longer required under the DLM rule and can be groomed.  However, these are retained because they are still protected by the their retention periods.  Backup set Diff15 will only expire at 2 a.m. on the Friday of Week 3.  However, backup set Diff15 is only dependent on backup set F1.  Thus  by Friday on Week 3, backup sets Diff11, Diff12, Diff13 and Diff14 would have expired and would have been groomed.

In this scenario, there should be sufficient disk space for hold the backup sets for Weeks 2 & 3, plus backup sets F1 and Diff15.

 

Is DLM an unnecessary complication?

You may think so if you got lost in some of my scenarios above, but this is not so. On the contrary, DLM is another layer of protection against setting inappropriate retention periods which do not adequately protect a backup chain.  For example, the retention period for differential backup are set so short that by the end of the week, the Monday's differential backup set has be overwritten and the subsequent differential backups are rendered useless.  DLM will protect against this happening.

If you are trying to squeeze every bit of space on your disk, then you have to sit down with a pen and paper and figure out like above what backup sets are retain on your disk.  Otherwise, get a disk with lots of space.

It does not mean that in all cases, DLM needs more disk space.  In some situation, there is actually a savings.  Take the first case above (Full backup on Sunday and incremental backups on Monday to Friday), as we have seen, with just DLM, the backup sets F1, Incr11, Incr12, Incr13, Incr14 and Incr15 can be groomed after backup job F2 runs and the disk space occupied by them would be freed.  Without DLM, we would have to use retention period to prevent them from being overwritten.  At the minimum, you would have to retain them for 1 week.  After backup job F2 runs, only backup set F1 can be overwritten.  Backup sets Incr11, Incr12, Incr13, Incr14, Incr15 would still be protected and cannot be overwritten, although they are useless without backup set F1.

Comments 9 CommentsJump to latest comment

David Palmerston's picture

First, Thanks very much PKH for your description of how the DLM process is working in BE2012 in more detail.

Second, as you mention early in your post - "This makes it difficult to visualise what is on your disk and visualising what is on your disk is critical when you are trying to squeeze as much as possible onto your disk"

I'm wondering if someone at BE and some of us in the community can put together a methodology for determining why "expired" .BKFs are still hanging around (and which server that they are protecting)?

Some possible reasons that I've come across already:

  1. Valid, legitimate backup that hasn't had another more recent disk backup completed.
  2. Renamed and/or retired server that is not currently in use.
  3. Incomplete and/or unsuccessful backup.
  4. A long retention period on a duplicate to tape that duplicated this B2D??
  5. One-time backup???
  6. Incorrect retention period set by operator at backup creation.
  7. Possible incorrect database and/or catalog links.

I'd love to be able to run a powershell command (or use something in the BE UI) into which I put a particular .bkf (or collection of .bkf names) and the output is a collection of server names that are being protected by the named .bkf files.

With tb of data, tens of servers, and thousands of .bkf files, I'm not very excited about looking for what recovery chain is preventing those 6-month old .bkf files from being deleted, nor does across-the-board deletion fit my governance policies. 

Some easy tool to backtrack through the database and point directly to the server whose recovery chain is being protected would be an enhancement for our operations.

 

+1
Login to vote
David Palmerston's picture

Further investigation into reasons has brought more info:

While working with tech support on a related case, they are finding in their testing the following:

1) When B2D storage is above the thresholds, DLM may NOT recover expired BKF space

2) One-time backups and duplicates to tape should NOT become part of the recovery chain

Anybody else got skills or interest in a tool to identify the server being protected by a recover chain based upon an existing BKF??

 

 

0
Login to vote
Sputnik's picture

Yes... I have a two week retention, with weekly backups, but I am seeing server data over a month old kept on disk.  We're backing up a lot of servers and terrabytes of data.  Getting rather frustrated with a full disk dedupe unit and having to manually delete Backup Sets.

Also, due to number of Backup Sets, BE2012 seems to crash sometimes whilst browsing the Backup Sets despite having a dedicated server for this with 16GB RAM.

0
Login to vote
David Palmerston's picture

Sputnik-

The Management GUI crash while browsing is a known and escalated issue with BE2012 SP1a - I'm hoping for a fix soon because it is just another productivity killer..  Fortunately it doesn't seem to involve any data issues (at this point)...

On the other part of this thread, we have been totally unable to get support excited about the DLM issues and hope that someone in engineering will see these threads and help us out...

+1
Login to vote
David Palmerston's picture

From my Jul 23 post, the case I was working with support on the BE Management GUI crashing while browsing seems to have been fixed at my site by installing the HF180964...

One step closer to nirvana!

 

0
Login to vote
David Grayston's picture

Thanks for this post regarding DLM - very helpful! I was stuggling with this new feature and trying to squeeze our backups into really not enough disk space.  You confirmed what I finally came upon was that you needed enough space for 2 full backups + all incrementals since the first backup and then only if retention period was set such that the last incremental was expired would the first 1 full + incrementals be groomed.

For us keeping the last recovery point on disk isn't so important because all backup jobs are duplicated to tape. I don't find that DLM takes this into account which I guess is understandable. But your linked Tech article, http://www.symantec.com/business/support/index?page=content&id=TECH187957 that with SP1a provides a registry entry to override and allow for the grooming of the last recovery point.

I think it would be nice if this was made as GUI config option as it seems many folks follow a Disk -> Tape duplication of jobs. Then retaining the tapes for much longer than the disk based backup jobs.

+1
Login to vote
Kiran Bandi's picture

Thanks for the Article!

For example, the retention period for differential backup are set so short that by the end of the week, the Monday's differential backup set has be overwritten and the subsequent differential backups are rendered useless.

The statement is true for incremental backups, but not for differentials.

0
Login to vote
pkh's picture

Thanks for spotting the error.  I had meant to say incremental.

0
Login to vote
adler.ravaioli's picture

Hi,

scenario:

Full backup B2D on friday at 8pm (Keep for 4 days option) than duplicate Full to Tape

In BackupSets page I see Expiration Date for Storage Disk correctly after 4 days, BUT Expiration Date for Storage Tape after 11 days.

Diff backup B2D from mon-thur at 8pm (Keep for 4 days option) than duplicate Diff to Tape...

In BackupSets page I see Expiration Date for Storage Disk correctly after 4 days, BUT Expiration Date for Storage Tape after 11 days.

How is "Expiration Date for Storage Tape" calculated???

DLM can't remove correctly Storage to disk and delivery free space on volume disk use for backup, until this Expiration Date for Storage Tape is ...