Video Screencast Help
Symantec to Separate Into Two Focused, Industry-Leading Technology Companies. Learn more.

Slow client side dedupe with Data Domain

Created: 19 Mar 2013 | 6 comments

I've been trying to get this client-side dedupe working correctly.  I did get it finally after much tinkering and reading some other forum posts, but now I am noticing that it is very slow.  Between 500-700MB/minute is what is reported in backup exec.  That test box I am backing up is a VM, with 1 processor and 3GB of ram.  I know the process is working because on the test VM, the network load is low, and CPU hovers around 60% or so druing the backup.  Also, on the Data Domain GUI, I can see the connection to DD Boost from that test VM.

So, I'm not sure what the problem is here with performance.  I would expect 2,000MB/min or better using this.  Anyone else have sucess with the same situation?

Operating Systems:

Comments 6 CommentsJump to latest comment

teiva-boy's picture

Never assume a certain throughput number.  You will be disappointed. ;-)

That said, your mileage will vary, and there is no hard/fast rule on how fast a backup should go. A virtual machine backing up just "file data," I'm already thinking your speeds are normal IMO.

If you were backing up a database like SQL, I would expect something in the thousands of MB/min.  If you had a physical server with SQL, I would expect even more.

As it stands you have so many components not tested and evaluated for performance like disk subsystem IOPS, network throughput over a shared NIC, CPU allocation, etc.  And that is just for the virtual environment.

There is an online portal, save yourself the long hold times. Create ticket online, then call in with ticket # in hand :-) http://mysupport.symantec.com "We backup data to restore, we don't backup data just to back it up."

Moltron1's picture

The phyiscal and the VM boxes I tested with had varying throughputs. I opened a case with Data Domain who after checking out their appliance and the settings in Backup Exec said that it could be a Symantec problem as everything checks out on their end.

Is anyone else out there doing this or am I plowing new territory?

teiva-boy's picture

I could show you screen shots of my customer environment coming near saturating a 10GbE link with multiple 7000MB/min jobs with a DD670...  But it would only make you feel inadequate.   wink

There are a handful of us on this forum doing it, and I've got dozens of customers doing it.  

Even in it's most basic config, I've seen numbers higher than what 1GbE is capable of producing, and that is the "BOOST," function working.  But again so many variables are needed to get there from disk to network, to host properties.

There is an online portal, save yourself the long hold times. Create ticket online, then call in with ticket # in hand :-) http://mysupport.symantec.com "We backup data to restore, we don't backup data just to back it up."

Moltron1's picture

We've got 3 Data Domains, one of them a 670. We're already seeing 7000MB/min jobs on VMware over 8Gig FC SAN in some cases.

Its the physical boxes we have still that are a pain. Some have lots of data that does not change much, yet I'm seeing slower speeds than a normal network backup.

Example: A physical server running a Quad 2.6 Ghz AMD proc with a 1gig network, 4GB RAM, and mostly SAS disks will do client-side dedupe at 1700MB/min for 17 hours to our DD670 on the same network. This is with several fulls already existing from the same server.

So, you have lots of customers doing this. Give me an example. I'm really just looking to compare mine to something else.

teiva-boy's picture

AMD...  I think thats the issue.  Make two systems, one Intel the other AMD, same RAM and disk layout.  I'll bet the Intel system deduplicates faster in every case.

There is an online portal, save yourself the long hold times. Create ticket online, then call in with ticket # in hand :-) http://mysupport.symantec.com "We backup data to restore, we don't backup data just to back it up."

Moltron1's picture

Here's another example.  I have a VM server with 4 CPUs (Intel Xeon E5649), 8GB Ram and has (6) 1 Gig ports LACP bundled through VMware to test this with.  Basically plenty of resources.

First backup (Server-Side Dedupe to Data Domain):

Elapsed Time: 2:27
Byte Count: 4.01 GB
Job Rate 1,923.00 MB/min

Second backup (Client-Side Dedupe to Data Domain):

Elapsed Time: 4:19
Byte Count: 4.01 GB
Job Rate 1,043.00 MB/min

BE Support has said that there are no setting to tweak for this and it depends on bandwith resources and the OST plugin.

Since I'm using a server with lots of resources available, and the latest OST plugin - I guess I have to conclude that this feature is poorly implimented.