Video Screencast Help
Symantec to Separate Into Two Focused, Industry-Leading Technology Companies. Learn more.

NDMP Incremental backing up unchanged data from IBM SONAS

Created: 05 Mar 2013 • Updated: 06 Mar 2013 | 4 comments
This issue has been solved. See solution.

Gidday,

Issue: NDMP Incremental backing up unchanged data from IBM SONAS

NBU: 7.5.0.3 Enterprise

One Master server (Win 2k3 R2), one Media server (Win 2k8 R2 SP1)

IBM SONAS gateway with NDMP v4 to XIV storage

We have an IBM SONAS remote NDMP client. All policies are configured for a monthly full backups to tape and daily differential incremental backups to DSU.

I have an issue where the differential backups are regularly backing up data that has not changed. It will regularly backup the same number of files and kb for days at a time on unchanging data. This means we are getting many TB more data backed up each day than we should.

For example:

Date          Start         Files  KB            Policy Type
2/03/2013     8:49:00 a.m.  3364   247514624     Differential-Inc
1/03/2013     1:57:22 a.m.  2881   246118843     Differential-Inc
27/02/2013    9:22:34 p.m.  2881   246118843     Differential-Inc
27/02/2013    3:31:15 a.m.  2881   246118843     Differential-Inc
26/02/2013    3:44:27 a.m.  2881   246118843     Differential-Inc
25/02/2013    1:09:34 a.m.  2881   246118843     Differential-Inc

This happens in multiple policies on multiple occasions, but apparently not consistently. It appears to act like a cumulative rather than incremental…

I've been attempting to determine if this is something to do with the NDMP levels (http://www.symantec.com/business/support/index?page=content&id=HOWTO53118 ). However, information is rather light on this subject. I assumed that NetBackup handled the levels in the background without having to pass variables in the policy? Theoretically it should be a level 2 backup each day, correct? Or is it incrementing each day up to level 9 and then having a fit? Does this mean I can’t do daily incrementals for more than 9 days in a row? Confused…

Thanks in advance,

Matthew

Operating Systems:

Comments 4 CommentsJump to latest comment

watsons's picture

Try enable the ndmpagent logs on the media server to see if it sends the right "backup command" to your IBM SONAS filer, check for the parameter of level (level=0 is usually a full backup, level=2 is your incremental)

vxlogcfg -a -p NB -o 134 -s DebugLevel=6

The logs will be in \netbackup\logs\ndmpagent\51216-134-*.log

The above log level can be reset to original by:

vxlogcfg -a -p NB -o 134 -s DebugLevel=1

Hope it helps.

RLeon's picture

All policies are configured for a monthly full backups to tape and daily differential incremental backups to DSU.

For NDMP, if you are aiming to use differential instead of cumulative for all your incrementals, then you should do weekly fulls and daily differentials in between.

Or is it incrementing each day up to level 9 and then having a fit? Does this mean I can’t do daily incrementals for more than 9 days in a row?

Since NDMP backup level never goes higher than 9, this means incrementals 1 to 9 will act like proper differentials as you would expect them to be, but starting from the 10th incremental, all further incrementals will still use level 9 which means they will all act as cumulative incrementals referencing the 8th incremental backup.
Things could become confusing, that's why it is recommended to do weekly fulls (level 0) instead of monthly fulls.

This happens in multiple policies on multiple occasions, but apparently not consistently. It appears to act like a cumulative rather than incremental…

My explanation above explains why this is so.

Also, since you have multiple NDMP policies, please beware of the following from the Nbu NDMP guide:

Do not perform incremental backups of the same NDMP data from two different policies.
Incremental backups performed by one of the policies may be incomplete, because NDMP filers perform level-based incrementals instead of time-based incrementals.

Consider the following example:
Policy A does a full backup of /vol/vol1 (level 0).
Policy B then does a full backup of /vol/vol1 (level 0).
The filer now considers the policy B backup to be the last full (level 0) backup of /vol/vol1.
Policy A does an incremental backup of /vol/vol1 (level 1).
The policy A incremental backup captures only the data that changed since the full backup that was done by policy B.
The incremental backup misses any changes that occurred between the policy A full backup and the policy B full backup.

I've been attempting to determine if this is something to do with the NDMP levels However, information is rather light on this subject.

For NDMP details, it is probably best to consult NDMP documentations instead of Symantec.
Know that with each successive run of the same differential incremental schedule, the NDMP backup level increases by 1 (until it reaches 9).
With cumulative incremental scheduels, the level stays as 1 - So that it will always reference the level 0 (full) backup no matter how many iterations it completes. (This is also why cumulatives do not have that weekly full recommendation; you can cumulate to your heart's content if you want...)

NDMP backup levels and backup schedules work like this:
A full schedule on Saturday and a cumulative schedule for Sunday to Friday would mean weekly full (level 0) and daily cumulative (level 1).

A full schedule on Saturday and a differential schedule for Sunday to Friday would mean weekly full (level 0) and daily differential (levels 1 to 6).
NetBackup will automatically do level 1 on Sunday, level 2 on Monday, and so on, until it reaches level 6 on Friday, then the next backup on Saturday will be level 0 again (full). Then the cycle begins again.
 

SOLUTION
Burpythehippo's picture

Hi RLeon,

Absolute gold, thank you!

Since NDMP backup level never goes higher than 9, this means incrementals 1 to 9 will act like proper differentials as you would expect them to be, but starting from the 10th incremental, all further incrementals will still use level 9 which means they will all act as cumulative incrementals referencing the 8th incremental backup.

This would exactly explain wht I've been seeing. It looks like I end up having daily cumulative backups 8 days after a full.

The problem I have is capacity - I simply can't sustain a weekly full schedule at this stage. So... the question is, would the following schedule work ok?

  • Monthly full
  • Daily differential (1 week retention)
  • Weekly Cumulative (4 week retention)

Would the weekly cumulative schedule reset the daily differential level back to 2?

Thanks,

Matthew

Burpythehippo's picture

Never mind, found the answer

A "Full" backup and a "Cumulative" backup can reset the dump level and the differential backups will always begin from the new dump level.

Thanks for your help folks. I'll mark this as resolved pending testing.

Matthew