Video Screencast Help

MSDP - can it be fooled ?

Created: 18 Apr 2014 • Updated: 14 May 2014 | 4 comments
This issue has been solved. See solution.

Hello Guys,

I am in a sitauation..i have 1 TB of MSDP pool configured.

Now,, i have a client to backup having around 2 TB of data.

And i was wondering if i can back it up ?

 >If i use multistreaming (say 4 streams of 500 GB's each) and send ONLY one stream to MSDP for a initial full and then keep sending INC of that stream would it will start doing de-duping it uptill a point where it is just backing up says Few GB's ? Now why i am doing this is that i can do this one by one for remaining streams going forward untill i have all the 2TB data backedup (in form of de-duped few GB's!) on that 1 TB MSDP ?

                                                                                                                                            -is it posible ?

let me know what you guys think....

Operating Systems:

Comments 4 CommentsJump to latest comment

RonCaplinger's picture

You cannot "fool" the deduplication process.  Your plan won't save you anything more.

Deduplication is at a block level, not a file level, so you can send the entire 2TB to the MSDP at once and see if it compresses enough. 

If you are already backing up your 2TB of data to tape, you should be able to see how well it compresses by noting the amount of data stored on a tape.  Deduplication can increase that compression ration, but it should give you an idea of how much space you will need in the MSDP.  And remember that in deduplication, any new data added to the original 2TB in the future would likely not deduplicate nearly as well.  It *might*, but it probably will be less space savings in relation to the original data.

SymTerry's picture

As with any compression, you never know how its going to compress till you do it. Yes I imagine that you will see some, but this will depend on the type of data your deduping. You may end up needing more storage. 

nbuno's picture

guys, i have some more thing to add here if you guys are so hellbent that we cant fool MSDP : ) ..since we are talking here about the first backup so anyways it is not gonna compress it (atleast first will be a full as per my understanding)..

SymTerry's picture

we are talking here about the first backup so anyways it is not gonna compress it

Actually no. even on the first backup there is deduplication. Look at HOWTO36304

The first backup may have a 0% deduplication rate; however, a 0% deduplication rate is unlikely. Zero percent means that all file segments in the backup data are unique.

The fingerprint cache is being created, and therfore dedupes what it can.  You have 2 TB of data to dedupe, so on the first backup of that data, just to fit it on a 1TB MSDP pool, you need half of the file segments to be the same. 

The real question is, how much new data is produced on every backup after?