Video Screencast Help

Differential Full Backup

Created: 21 Dec 2013 • Updated: 03 Feb 2014 | 11 comments
SHANKM's picture
0 Agree
4 Disagree
-4 4 Votes
Login to vote

According to me, there should be a concept like “Differential_Full_Backup” in every Backup Technology, which will make “RESET” all the data set of “Last Differential Backup” (cumulative incremental backup).
I mean to say, after that backup, if someone takes a Normal Differential Backup (cumulative incremental), it will just take a backup of latest change (after having the 'Differential_Full_Backup') in that particular volume.

The benefit of this backup is we can run this backup (Differential_Full_Backup) in a span of certain time frame (i.e 'every weekend') with a Little High Retention Period (Product Specific) instead of a complete “Full Backup”, and after that a Fresh New 'Incremental (differential incremental) or Differential (cumulative incremental)' Backup could be possible......

Please let me correct, if my logic/concept make you feel wrong.....

Comments 11 CommentsJump to latest comment

SHANKM's picture

There is nothing much to do, to make it happen. Just have to track the Last Time Stamp of ‘Full_Backup’ and the ‘Differential_Full_Backup’ (itself), and the same way 'Incremental (Differential Incremental) and Differential (Cumulative Incremental)' will track the last Full_Backup & Differential_Full_Backup as well.

But we can add few more things over here, like an example "Differential_Full_Backup runs in a span of 7/15 days, few modified files got backed up in the first two set of Differential (cumulative incremental) Backup and then somehow it’s got deleted from the volume, so in the next 5/13 sets it will be not getting backed up". But, if the “Differential_Full_Backup” compare the catalog of itself with the all previous Differential (cumulative incremental) backup and throws an alert “these many file got missed in the backup, you may copy it from the last Backup”. How it is…?  

I mean it will also track all the files are getting backed-up (modified and newly created), since the last Full_Backup or Differential_Full_Backup ran, just by the comparing catalog.

0
Login to vote
SHANKM's picture

Comparing catalog should be a part of the only Differential_Full_Backup set, not in every single Differential_Backup set. The logic behind the comparing catalog is very simple, “every newer version of catalog will be getting compared with the older one”. Entry of each & every file along with the path details from older version catalog, has to be present in the following newer version of catalog.

First itself (Differential_Full_Backup catalog), will be getting compare with the just previous backup (Differential_Backup) set, and then one by one all the Differential_Backup set, till the day last 'Full_Backup' or last 'Differential_Full_Backup' got ran (in reverse order of date counting).

0
Login to vote
SHANKM's picture

Now this one is the “Master Stroke”, but little bit Complicated….

This idea is related to ‘Carry Forward’ the all those deleting files which got copied in the Differential_Full_Backup’ set, to a new “Full_Backupset. But this feature should be optional (you can activate this feature on behalf of your requirement).

Let’s guess our Differential_Full_Backup runs every weekend with the 4 Months Retention, and a Full_Backup (with the help of this feature) runs every quarter of the year (obviously with the Higher Retention as compared to Differential_Full_Backup, Ex- 2 Years). Now if we can maintain a separate list/catalog for all deleting files, which was being carried from the Daily Differential_Backup set to Differential_Full_Backup set in every weekend, and let it be on the server unless the data got expired (here it’s 4 months), so that we can copy all those deleted files (last stage of the each & every deleting files, same way by comparing list/catalog along with the path details) in a complete Full_Backup set, which got a Higher Retention period. Completing a successful Full Backup will remove all the previous list (every weekend, for last three months) of copying files (deleted files) in all Differential_Full_Backup set from the Daily Differential_Backup, regardless the retention period (4 months) of the data. But only the list/catalog, not the actual data.

Remember, without having any previous Differential_Full_Backup set, this backup doesn’t have any such difference with a Normal Full Backup.

 

Now let’s say, if someone wants to make run a Normal Full Backup (without enabling this feature) between this time frame (i.e ‘every month end’) in order to more safety aspect, with a little higher retention period (7 Months) but lesser than Quarterly Full Backup retention, it will not allow to remove the list/catalog of deleting files and got copied of last four/five Differential_Full_Backup set.

0
Login to vote
SHANKM's picture

But still there are some loophole in this Idea/Planning, let's say some modified file got deleted after having a successful weekly Differential_Full_Backup, which was not modified anymore, so naturally it will not come into the daily Differential_Backup set.

In this case how it (Deleted Files) will get the retention of Full_Backup (2 years) ........!!!!!???????broken heart

0
Login to vote
SHANKM's picture

Ding, Dong….. Time of Restoration…..

Let’s guess we are only having a combination of Daily Differential_Backup (15 Days Retention), Weekly Differential_Full_Backup (4 Months Retention) & Quarterly Monthly_Full_Backup by the help of “Special Feature” (2 Years Retention).

Now somehow after two months, two weeks, three days one filesystem got crashed, so how we will restore it?

1. Restore the last full Backup set of the specific file system from Quarterly Backup.

Now I would like to introduce a new feature of data restoration from Differential_Backup.

Whenever we will be going to initiate any restoration from Differential_Daily/Differential_Full Backup set, and if it contains more than a single individual file (A Folder/Filesystem), it will ask you one thing [1st Restoration, or Compare Restoration], you can ignore this thing if you don’t need, but if you click over the checkbox (use this feature), the list of restored data along with the path detail will be getting save on the Backup Server (by default for 3 days, or you also can specify the number of dates as much as you require).

2. Restore the last successful daily Diffrential_Backup set of the respected filesystem, with the option 1st Restoration by overwriting.

3. Restore all the weekly Differential_Full_Backup set of the respected filesystem for last two weeks & two months one by one, with the option Compare Restoration by overwriting in reverse order of date counting.

Conclusion: - 1st Full backup restoration is very simple, 2nd overwriting the full backup with the latest daily Diffrential_Backup set by the option 1st Restoration, so that last three days got secure and the list (filename along with path) of data which got restored, will be getting  ignored in the next restoration.

By using the option Compare Restoration we just have to restore the all weekly Differential_Full_Backup set, only the remaining data will be getting restore and again the list of restore data will be getting merged with the first restore list (Daily Diffrential_Backup) and it will just keep continuing until we reached in the date of last full backup, which got restored first (Two Weeks & Two Months back).

0
Login to vote
CraigV's picture

...Is this an Idea or an essay???

I see you also posted this on Facebook to IBM Research...?

 

Alternative ways to access Backup Exec Technical Support:

https://www-secure.symantec.com/connect/blogs/alte...

0
Login to vote
SHANKM's picture

Initially every idea seems like an Essay, unless anyone can make it proof by the execution in Real life scenario. Anyway the first one is “Core Idea”, and all the comments are just “feature of Idea”.

0
Login to vote
CraigV's picture

...so are you posting the same idea to every backup vendor?

Alternative ways to access Backup Exec Technical Support:

https://www-secure.symantec.com/connect/blogs/alte...

0
Login to vote
SHANKM's picture

I posted it because the 'core idea' is generic, not any product specific, might be the maximum possibility to make it “REAL” with the NetBackup only (as of now), but it could be integrated by any enterprise level backup software with a little modification of Main Software.

0
Login to vote
SHANKM's picture

Here is something about Best Practice…….

To get a better performance (In aspect of Faster Operation) recommendation is taking backup over Disk, at least the “Daily Diffrential_Backup”, it’s not that much difficult because of ‘Less Retention’ (15 Days only). Now if the company belongs to Billion Dollar revenue family, I think they also can afford the “Weekly Differential_Full_Backup over the Disk as well, but if the company is very clever, like Japanese (who knows very well, how to reduce the ‘WASTE’.....), surely they will implement a “Deduplication solution based any VTL”…….

So only the “Quarterly Full_Backup, which will go over the Tape Media, and I don’t think it will be a good idea to get involved any Deduplication Method (Source Base/Target Base) over there, as the retention period is pretty high (2 Years). Anyway just 'Four Times' in a Year, so it could be considerable from any type of Budget aspect.

0
Login to vote
CraigV's picture

...I'm sorry but the Ideas section to my mind isn't something for this. Either blog it, or stop it. I am sure Symantgec, as a billion dollar revenue company has considered a LOT of options for their software.

This is becoming rambling now...all Ideas should be short and to the point in order to be understood better.

Thanks!

Alternative ways to access Backup Exec Technical Support:

https://www-secure.symantec.com/connect/blogs/alte...

0
Login to vote