Video Screencast Help

BE 2012 and Data Domain w/ DDBoost

Created: 25 Aug 2012 • Updated: 03 Sep 2012 | 32 comments

Background:  So we have Backup Exec 2012 and two Data Domain DD620's.  We have all the required licensing including the Data Domain DD Boost and replication licenses.  I have installed the Open Storage Technology (OST) plugin 2.5.0.3 as suggested for the version of BE.  Both DD boxes are visable and connected to my BE server.  I have foud some documentation and a video that explains some of what I am looking for but not entirely and most of it is for BE2010.  The BE 2012 documentation from DD is not up to date yet.  I am trying to setup backups as my tape drive has had enough and is not working properly.  Anyone I have talked to just says you backup to it, then I continue finding little things that need to be done.  No verify, no precompression, some said no deduplication in the job, other said server side dedupe because DDBoost manages it with OST plugin.  Too much information and not enough people knowing what to tell me.  The DD weekend support guys don't know the software titles they just know the hardware.  I need help now unfortunately...

Question:  What settings do I use for Backups?  I have Active Directory, Exchange, SharePoint, and SQL agents.  

 

Comments 32 CommentsJump to latest comment

pkh's picture

Do you have the Dedup licence installed on your media server?  If so, you can just use the Add Storage wizard to add your device and then backup to it.  It will take care of the deduplication for you.  You don't have to define a dedup folder.  The rest would be just normal job setup.

CraigV's picture

...just be aware that BE won't support DD Boost...this is a functionality of EMC NetWorker. I suspect you just got sold functionality on the Data Domain you won't be able to use.

Stream your servers off as individual backups at the same time or staggered if need-be. Your network speed would dictate how fast you can get your data to the DD.

Alternative ways to access Backup Exec Technical Support:

https://www-secure.symantec.com/connect/blogs/alte...

CRoth2's picture

You couldn't be more wrong with your first comment.  EMC Data Domain Boost Compatibility Matrix (8/1/2012 includes BE 2012.  3/10/2011 includes BE 2010 versions).  Backup Exec 2012 is DDBoost compatible with OST plugin 2.5.0.3 (Restricted Availability currently, contact support).  Backup Exec 2010 is DDBoost compatible with OST versions BE R3 (2.4.1.0, 2.3.3.0, 2.3.1.0), BE R2 (2.3.3.0, 2.3.1.0, 2.2.3.0, 2.2.2.0, 2.2.1.1) depending on you DD OS version.

Please don't confuse people with false information.  DD Boost is fully supported with Symantec Backup Exec as well as EMC Avamar, EMC NetWorker, EMC Greenplum, Quest vRanger Pro, Symantec NetBackup, and DD Boost for Oracle RMAN.

CraigV's picture

It's not a case of me confusing people with false information dude...I was told that by an EMC DD engineer here in my company. So I need to go back to them and ask them for details on what products do support DD Boost, as apparently it was only NetWorker that supported it properly.

Thanks!

Alternative ways to access Backup Exec Technical Support:

https://www-secure.symantec.com/connect/blogs/alte...

CRoth2's picture

Sorry I was a bit overt there.  Have your peeps update you cause many more than NetWorker are supported at this time.

CraigV's picture

Haha...not fussed, & cool. I will do that! Funny how a vendor tries to push all their products on you :)

Alternative ways to access Backup Exec Technical Support:

https://www-secure.symantec.com/connect/blogs/alte...

teiva-boy's picture

NetBackup, BackupExec, and NetWorker all support BOOST.  (among others as mentioned, like Oracle which is wicked cool)

Netbackup is media server only.

BE and NW can do client-direct and media server dedupe prior to hitting the DD.

However there are a few things that OST in general doesn't support widely or at all like NBU A.I.R, NBU Optimized Synthetics, etc..  These are items I think Symantec keeps to itself as a competitive edge.

Even Quantum's DXi has a BOOST like feature too.

 

OST is Symantec's term.  BOOST is EMC's.  But BOOST is really just an extension of OST.  Symantec will say they don't support boost in many cases, but ask them about OST and they'll back peddle and trip over themselves.  Sometimes the field SE's can be too sales'ie and just flat out lie or mislead you.  EMC included...

There is an online portal, save yourself the long hold times. Create ticket online, then call in with ticket # in hand :-) http://mysupport.symantec.com "We backup data to restore, we don't backup data just to back it up."

CRoth2's picture

Yes we have deduplication license.  I created a new "Backup to Deduplication Disk Storage and then Duplicate to Deduplication Disk Storage" job.  I chose no verify, no encryption, no (pre)compression (doesn't even show (pre)compression once you select the DD as the backup device).  However once the DD is selected I get two options on the storage tab:

1)  Enable the remote computer to directly access the storage device and to perform client side deduplication, if it is supported

2)  (DEFAULT SELECTION)  Enable the remote computer to access the storage device through the Backup Exec server and to perform Backup Exec server-side deduplication, if it is supported

I left it with the default for my first run through.  Jobs backed up successfully and much quicker than my old tape backup for sure.  However, I can see where allowing the remote clients to go straight to the DD box could increase my throughput as the DD has dual network connections and the Backup server only has one, making it the bottleneck.

CraigV's picture

So what were your backup speeds? You should also get better throughput if you stream off more jobs to the DD. Just work out where the "sweet spot" of that is so you don't stream too much across your LAN.

Alternative ways to access Backup Exec Technical Support:

https://www-secure.symantec.com/connect/blogs/alte...

CRoth2's picture

Well the jobs finished in 3 hours.  With our previous mix of direct to tape and to disk dupe to tape jobs, it would take approx 16 - 18 hours.  Dupes were always still running into production time, but load was off servers at least...  The time change is amazing!

I currently have 16 connections allowed to the Data Domain storage, which I was told is the max by DD.  I may have to cut down on the number because the media server was pegged at 90%-100% network bandwidth use and 70%+ CPU usage.

CraigV's picture

...have a smile on my face checking those stats out, and I guess so do you! :)

Not sure what your specs are like on the media server, but an upgrade can always be considered too. Not much else to do there but check that switch ports and NICs on the servers are all hard-coded to the fastest speed of the NIC, and to make sure that any AV isn't actively scanning your BE services.

Otherwise that looks good...well done!

Alternative ways to access Backup Exec Technical Support:

https://www-secure.symantec.com/connect/blogs/alte...

pkh's picture

You should not use software compression or encryption with your job because this compresses or encrypts your data before the deduplication process and compressed or encrypted data do not dedup well.

You should use client-side dedup to avoid the bottle-neck at the media server.

CRoth2's picture

OK, good to hear about the compressing and encrypting, because I am doing neither of them.

As I mentioned in my reply to CraigV, the media server really was pegged down last night for network and cpu.  So much that I couldn't really use the admin console of Backup Exec on the Jobs view.  It keep saying not responding when I'd try to click anything.

I am thinking I need to use the client side dedupe process, but it may be better to try just lower the number of connections to the DD box first.  It is a much less time consuming change, will not make me change any job definitions, and it will obviously still be done well before my old tapes were. 

 

After I get that squared away I can try the remote dedupe.  Does that really take significant load off the server?  Does the remote agent then send deduplicated data directly to the DD box or does the deduplicated data still get sent to the media server and the media server sends it to the DD box?

CraigV's picture

...using client-side deduplication, the media server is completely bypassed. You might be able to run some of the servers through the media (say smaller servers), and have your larger servers pass data straight to the DD itself.

Check below:

http://www.symantec.com/docs/HOWTO23347

Thanks!

Alternative ways to access Backup Exec Technical Support:

https://www-secure.symantec.com/connect/blogs/alte...

pkh's picture

In simplified terms, the dedup process goes like this.

1) take the data and chop it up to small chunks

2) hash the chunks and send it to the dedup folder/device

3) if it is a unique chunk, send the chunk to the dedup folder/device

With server-side dedup, all these is done on the server.  With client-side dedup, these are done on the client.  When I say send to the dedup folder/device, the data will still be send to the media server because the dedup folder/device is connect to it, but there will be no processing done on the media server.

Ichris's picture

I understand the concept of what you are noting, but do you need to (if its possible) install the EMC OST Plug-in on the Client Server, so Client-Side deduplication works? I'm using BE2012 with SP1a, with all the needed licenses via Symantec for BE2012, backing up MS 2003 R2 Exchange (system partition, not data)and 2008 R2 AD Server (both with BE2012 Agent for Windows licensing, and both physical serves) to a EMC DD630 with DDboost licensing. BE2012 backups servers to defined OST Storage successfully with exceptions, "Client-side deduplication is enabled for this job, but it could not be used." Any thoughts?

 

CRoth2's picture

I haven't tried the client side dedupe yet, but you definately need the OST from Data Domain on the media server.  Are you asking if the plugin is needed on the remote client servers as well?  I don't know the answer to that...  Are you using the 2.5.0.3 version of the plugin?  It is not general availability (GA) yet and is currently restricted availability (RA).  You need to get this from support as it does not show in the downloads section of my.datadomain.com. 

I can try one and see if I get that also.  Maybe the remote agents need to be repushed after the OST is installed?  Can anyone comment?

Ichris's picture

Thank you for your reply. We have OST Driver (OST_2.4.1.0-289644_OT_windows) installed on BE2012 SP1A CASO and MMS Servers. In regards to repushing the Agent for Windows after the OST install, this has been done. As you noted in the beginning, BE2010 documentation doesn't quite line up with what I'm looking at on a BE2012 server. It may be there - but the steps are very different and I've been looking.

teiva-boy's picture

Finally someone else that is seeing the awesome speeds of using an OST device and not the native dedupe in BackupExec!   I should post a screenshot of my 5000-7000MB/min speeds on every day backups, using BE2010 and BOOST.

For client direct, YES the OST client needs to be on the client.  Though there are random issues where the job log says it couldn't do client direct and defaulted back to media server dedupe.

Also to note, the clients have to have connectivity to the DD box too.  Lots of folks use a dedicated VLAN or direct connect.  

 

If you use client-direct you will see your media server CPU usage drop down, and now move to the clients that are doing the work.  This is good and bad, so try it on servers that are not heavily used first.  servers that routinely go over 40% CPU, perhaps avoid.

 

There is an online portal, save yourself the long hold times. Create ticket online, then call in with ticket # in hand :-) http://mysupport.symantec.com "We backup data to restore, we don't backup data just to back it up."

Ichris's picture

Teiva-Boy - Thank you. With your comments and with Symantec Support Help, my backups with deduplication now works.

By installing the EMC DD630 associated OST client ON the servers I'm backing up, initiating a BE2012 client update on my 2003 and 2008 R2 servers, removing associated shared direct storage from the CASO and MMS servers, re-adding the associated direct stoage to the CASE and MMS servers, restarting the BE2012 services on both the CASE and MMS servers, restarting the client services on the servers being backup, I am now able to run client deduplication (2003 and 2008 R2 Serves) to EMC DD630 with significant speeds.

Thank you!

CraigV's picture

...I marked this as the solution, but for interest's sake, what are your speeds now?

Alternative ways to access Backup Exec Technical Support:

https://www-secure.symantec.com/connect/blogs/alte...

CRoth2's picture

I'd appreciate the time to test his suggestion before a post be marked as solution.  I am going to try installing the OST on the client.

CraigV's picture

...sorry, it slipped my mind you were the OP! Solution removed!

Alternative ways to access Backup Exec Technical Support:

https://www-secure.symantec.com/connect/blogs/alte...

CRoth2's picture

Thanks.  I just don't want people to stop looking at this when I haven't confirmed it working.  In aditiona it seems the poster above did a ton of removes and installs.  Seems to me that not all of them should be necessary.  Especially removing the DD from the Media server (if I read that correctly), that seems beyond necessary, but maybe it is.  I want to methodically try things one at a time.

The below document all seem to be for 2010:

This document says, "The remote computer must be configured as a Remote Agent for Deduplication."  
http://www.symantec.com/business/support/index?page=content&id=HOWTO23347

This document gives the general overall how to:   
http://www.symantec.com/business/support/index?page=content&id=HOWTO23346#v26143081

This document gives instructions for Configuring a remote agent for deduplication
http://www.symantec.com/business/support/index?page=content&id=HOWTO23345#v26143099

Just found a start for where i can look for 2012 deduplication setup.  Going to try suggestions given in this thread while following the Tech docs.  I'll try and provide a minimal list of things that actually need to be done.
http://www.symantec.com/business/support/index?page=content&id=HOWTO73694

Ichris's picture

CRoth2 - I agree with you - I listed a lot of steps to getting the BE2012 client deduplication to work, and it should not be this way. I have look extensively for why it didn't work for me, but there just isn't the documentation out there to defined the grey areas. There is a lot of documentation pointing to BE 2010, but things look drastically different. If you can find a way to make this process easier, your insights would be appreciated. Please post what steps you come up with, so we can all benefit from what you learn. Thank you.

CraigV - I'll note my backup speeds have a few more days of testing.

CRoth2's picture

I have been working with Symantec Advanced Support team for Deduplication.  They said the process should look like:

  1. Load OST plugin on the remote server to be backed up
  2. Reboot the server after installation, required for the RAD (forget what stands for) to connect properly
  3. Change the job from server based dedupe to client based dedupe.  When on client dedupe you'll see two additonal options "Direct access properties" and "Direct access sharing"
  4. Likely requires media server BE services restart (said so in bottom left corner of BE console and in the "Direct access sharing")
  5. On "Direct access properties" I used "System Logon Account" (when using "ddboost" user it failed)
  6. Ran job and worked.

Both I have got working are Windows 2003 R2 SP2 servers.  I am going to try some of my 2008 R2 SP1 servers.

Note:  You can tell if things are working by going to the DD box web interface Data Management -> DD Boost -> Activities and view the server names that have connected.

Ichris's picture

CRoth2 -

Thank you for the steps you documented. I haven't had a chance to follow these steps, but they seem cleaner. I'm still looking into my next comment - but with Client-Side deduplication I'm seeing job rates between 2100-2600 MB/min and our HP ProCurve 2810G-48 processing at 10%-15%, in comparison to Server-Side processing of job rates of 3800-6000 MB/min and our HP ProCurve 2810G-48 processing between 85% -98% - with unacceptable side effects on the network.

These jobs rates look to be pretty consistent between backing up (with Client-Side Deduplication) a 2003 Exchange server (4GB memory/2x 2.4GHz processors) and 2008 R2 AD Server (24GB/8-Core 2.4GHz) to a EMC DD630; BE2012 Servers Specs (Caso/MMS) are16GB/2xE5620 2.4GHz processors).

Thank you for the work you're putting into researching this, and please updates us with anything else you find out.

Ichris

Ichris's picture

Clarification of above steps - Enabling Client-Side Deduplication to a EMC DataDomain:

  1. Install the appropriate DataDomain OST client ON the servers (physical servers) to be backed up.
  2. Initiating a BE2012 client update on my 2003 and 2008 R2 servers, reboot.
  3. Removing the associated shared direct storage from the CASO and MMS servers (go to Backup and Restore, select server being worked on, click on details, Direct Access Sharing, uncheck servers from storage units, apply).
  4. Re-add the associated direct storage to the CASE and MMS servers (undo step 3)
  5. Restart the BE2012 services on both the CASE and MMS servers (for me – I restart CASE first, then MMS to it can confirm its services with CASA)
  6. Restart the client services on the servers being backup (Service: Backup Exec Remote Agent for Windows)
  7. Now I am able to run client deduplication (2003 and 2008 R2 Serves) to EMC DD630 with significant speeds.

Ichris

 

CRoth2's picture

Same process worked on one of my 2008 R2 SP1.  Need to try som more.

CRoth2's picture

I have not added any others yet as I am troubleshooting one annoying one that won't cooperate.  Hopefully I can confirm this works for others.  Can anyone else confirm the process listed is working for them?

Riaz's picture

Hi

DD's come with built in de duplication technology. So I believe you dont have to worry about the de duplication stuff. You can look at the Pre Compression & Post Compression rate to find how much de duplication has occured.

DD Boost offloads a part of de duplication process to the Backup Exec server. Means less data will have to flow through the network from Backup Exec server to the DD device. Results in increased speed and less bandwidth utilization.

Most importantly, I have read somewhere that you have to rely on the de duplication technic either on the DD side or on symantec side. Meaning you should not use de dupe feature from both.

 

Regarding verification, i am also stuck with the verification taking long time to complete. I am still looking for a solution. If anything comes up I shall let you know. According to DD support, DD is more optimized to write than read. So reading takes some time. And so verification does. Not very easy to digest.

Thanks

Riaz

Ichris's picture

Riaz - If you install the deduplication client on the client, the deduplication process is completed from the client, and data is sent from the client to the EMC DD's - the Symantec Backup Exec CASO only has to keep track of the record keeping. IF you do it this way - your network switches will not be pushed to their switching capacity, and your Backup Servers are free to do other tasks. Please note - if you don't use client side deduplication - you're correct, data between the Backup Exec servers and the DD's is less, BUT the data from your client to your Backup Exec is 1-to-1, it's hasn't been processed, everything gets transfered over to your Backup Exec server for processing.

In regards to Microsoft Exchange - I read posting from both sides - server side vs client side deduplcation. I run client side deduplication and it works well.

In regards to Micrsoft SQL 2000 Enterprise and Microsoft 2005 Enterprize, client side deduplication works well.