Video Screencast Help
Symantec to Separate Into Two Focused, Industry-Leading Technology Companies. Learn more.

Slow performace from BKExec 2012 to DeDuplicated Disk

Created: 24 Sep 2012 | 17 comments

Hello,

We've just moved to BKExec 2012 from 2010. Fully removing then installing the software from scratch.

We have two BKExec servers at different sites, both the same hardware spec, Windows 2008 X64 R2, SP1 with 32 GB memory.

Our deduplication area is hosted on or Equalogic SAN connected by the LAN direct to the Backup Exec Server.

The problem we are having is that backup are slow, especially exchange. Backing up a 171gb information store took almost 8 hours and failed reporting '-1022 There is a disk IO error'

The backups speeds initially build up to around 900MB/s, but soon fall down to 390MB/s.

Backing up a file server (1.5TB) took 32 hours at a rate of 778MB/s

Performance wise, network on both server (backup and target) is never higher than 25% for both the network and san links. Memory usage is about 40% with cpu hardly being used.

With the file server, we have F-Secure, but we ran a pre and post job command to stop this from scanning while the backup was running.

Windows and BackUp Exec are both fully up to date.

Any help on this would be greatly received.

Thanks

Lee

Comments 17 CommentsJump to latest comment

Jaydeep S's picture

Performance issues can be tricky as it might be a combination of lots of potential causes. Please look through the article below as they are fairly elaborative about what could be done.
http://www.symantec.com/docs/TECH49521
http://www.symantec.com/docs/DOC5481

However, I would suggest you to to try to disable Check Point Restart. Backup Exec 2012 enables the Checkpoint Restart feature by default.  This can make the backup speed slower than previous versions
http://www.symantec.com/docs/TECH191072

Javed_Nawaz's picture

Hi leeweedon,

Any update from the last Engineers posting on your issue ?

CraigV's picture

Hi,

1. Yesterday was the 25th...give the OP a chance to test!

2. Please stop sending update requests in the forums...rather send a PM! Nothing new to add by doing so.

Thanks!

Alternative ways to access Backup Exec Technical Support:

https://www-secure.symantec.com/connect/blogs/alte...

leeweedon's picture

Thanks for the reply Jaydeep.

We have identified the problem to be the backups of the Information Stores. A flat file backup of the exchange server works at around 1000MB/SEC which is not ideal, but acceptable.

When backing up the IS using the agent it still runs less that 300MB/sec   We have tried disabling Verify, Test Run, Enable Checkpoint Restart but this doesn't change anything.

I've even tried without the consistancy checks being enabled, but still no improvement.

Our exchange server 2007 is running Windows Defender and ForeFront.

Regards

Lee

CraigV's picture

...is your AV perhaps scanning the dedupe folder?

Alternative ways to access Backup Exec Technical Support:

https://www-secure.symantec.com/connect/blogs/alte...

leeweedon's picture

Do you mean the AV on the BKEXEC server? If so, no. I removed the AV completely from it.

Regards

Lee

CraigV's picture

That's what I meant Lee. Is the AV on the Exchange server perhaps scanning the beremote.exe service? What happens to your backup of the Information Store if you disable the AV on the Exchange server?

Also, what is the configuration of your storage on which the dedupe folder is located?

Alternative ways to access Backup Exec Technical Support:

https://www-secure.symantec.com/connect/blogs/alte...

leeweedon's picture

I have just tested using a pre-job command to stop Windows Defender. This seems to make no difference, but I will test this fully this evening.

Regards

Lee

Jaydeep S's picture

leeweedon could you please try and run the Information store job to a test B2D folder or a Tape Device,instead of Deduplication folder/device. This will atleast isolate, if it is the transfer from source or th write to target is in fault.

Also would be nice if you can try the steps in http://www.symantec.com/docs/TECH186741

leeweedon's picture

Carrying on from this it doesn't appear to be AV related. 

Backing up to Local Dis Storage (non-DeDupe) is also very slow.

Regards

Lee

CraigV's picture

OK...let's go back to the start...you have 2 backup servers on separate sites. Are you backing up across a WAN link to your dedupe folder?

Alternative ways to access Backup Exec Technical Support:

https://www-secure.symantec.com/connect/blogs/alte...

leeweedon's picture

We do have two backup server, one at each site. The sites are connected by a 1GB LES. 

The backups we currently have setup do no go accross the sites, they are done by our server at site A to a de-dupe folder on SAN based on Site A.

Regards

Lee

CraigV's picture

OK great...so no WAN bottleneck with servers on Site B being backed up to Site A.

So if the backup to a normal B2D is slow, this rules out dedupe. The AV is ruled out as it isn't running.

What are the specs of your backup server? Are all your NICs and the switch ports they are connected too hard-coded to the fastest speed of the NIC? ie. 1000MB FULL?

Have you checked for any hardware failures, or hardware in prefailure?

Thanks!

Alternative ways to access Backup Exec Technical Support:

https://www-secure.symantec.com/connect/blogs/alte...

leeweedon's picture

The NIC has been configured in a team, dual 1000mbps, connecting to our Cisco core (both configured in a port-channel) 

I've got the latest drivers etc and all hardware apears to be working ok.

The only bottlenick I can think of is possibley local HDD read/write IO?

BK Exec servers are 64bit, 32GB RAM, Intel Xeon X5550 DUAL x QUAD CORE @2.67

Windows 7 Server 3008 R2 Standard.

Regards

Lee

leeweedon's picture

Backups from any server are running much better now we have teamed the NICs.

However, Exchange is still a killer. averaging 600MB/min.

Forefront security is running, but have tried to fully disable this but it makes no difference

Regards

Lee

Gurvinder Rait's picture

Recommend you to open a case with tech support. Also does this job fails aways ? and have you seen the Issue with both Full and Incremental to dedupe ?

pkh's picture

.Did you realise that the last answer to this discussion is almost 2 months old?