Video Screencast Help
Symantec to Separate Into Two Focused, Industry-Leading Technology Companies. Learn more.

Exchange 2010 backup REALLY slow

Created: 05 Jun 2012 | 13 comments

Hi all!

We have some customers complaining about how slow is Exchange Backup Jobs is with Backup Exec.

I have visited both of them and they were using the following (64 bit):

  • Exchange 2010 SP1 (both VMs).
  • BE 2012 to SATA DAS array / BE 2010 to FC 8GB SAN array
  • Gigabit network.

No tape autoloader / drives involved. We are backing up using RAWS agent instead of AVVI (We are not using AVVI yet because there were some trouble implementing it (but it seems I managed to go through it with some help from here)).

The Exchange backup job starts and stops normally. However, the reported backup rate is an awfully 88 to 100 MB (yeah, MEGABYTES) per minute. It does not happen to the file system on the same server (there, it jumps to a really awesome rate of 4900MB per minute).

There are no bottlenecks, nothing wrong with their network / server and I have run out of ideas why is it SO slow. No technotes, nothing in the forums.

Thanks a lot for your help!

Comments 13 CommentsJump to latest comment

Sush...'s picture

We have seen these issue previously but nothing that it could be reproduced. So I would suggest that if you have a Support contract with Symantec then open up a support case so that this issue can be thoroughly investigated.

Just before that ensure that the disk where the Exchange databases are located is not highly fragmented and that the destination (b2d device) is not fragmented too. Do check the event viewer on the Exchange server to see if there are any ESE events generated when the backups are running. Another troubleshooting step would be to turn off Consistency check of Exchange db in the backup job and see if that improves the speed. Note that we recommend to use the condidtency check option and turning it off is just a troubleshooting step.

Do let us know if this helps.

Thanks,

-Sush...

Hope this piece of Information Helps you... and if it does then mark this response as Solution....!!!

Marco Tafur - Qualitas Technology's picture

We will help our customer to open a support case for this. However, this is not the only one who has this problem. And it started with BE2010. BE12.5 customers (with Exchange 2007) are just fine.

Right now his full Exchange Backup takes more than 24 hours for just 200GB of data.

Disks are not fragmented at all (it is a new install of Exchange 2010 SP1 in this particular customer).Event viewer does not show any ESE events. I have not tested the consistency check option because I have no access to the VM yet. I'll test that and report results.

Another approach I'll try is to back up that VM using AVVI. It seems AVVI does not use Microsoft's VSS provider but Symantec's. I just happened to do a test with SQL server 2008 (it was reporting the same performance issues backing up through RAWS). With AVVI, it jumped from 100MB/min to 4900MB/min. However, the drawback is that more space is required on storage. And I haven't tried to do diferentials.

Thanks for your help. If you have another insight, please let me know.

Marco Tafur - Qualitas Technology's picture

We managed to get the AVVI working for Exchange 2010 and speed has increased in 400x.

However, there is still a nasty bug when we have the CAS and the MBOX in different servers.

Although we are doing fast backups now, it does not answer as to why Exchange backup is so slow.

We have tried to remove the consistency, but no joy. I have heard about some performance issues with the checkpoint restart option. Does anyone know what is it?

Sush...'s picture

Well, when CAS and Mbox roles are on different server then backing using AVVI is not supported. Its not a bug but by design. 

The Checkpoint restart issue with performance is just reently seen. You can try to uncheck the option for Checkpoint restart but I am not sure that it applies for Exchange backups too.

Thanks,

-Sush...

Hope this piece of Information Helps you... and if it does then mark this response as Solution....!!!

Marco Tafur - Qualitas Technology's picture

Hi Sush,

I actually ended up calling Symantec Support and they told me the following:

- I must restore using a redirection to the CAS server. 

I did it and voila...it worked. So my customer is now happy.

However, I disagree that this topology is not supported. Many of my customers have both the CAS and Mailbox roles in different servers.

I understand it is because of Exchange Web Services which are provided only by the CAS (and that's how the restores are done). I would like some extra config on BE to insert the CAS name, so restores  would be easier.

I haven't explored further about this issue as my customer is happy with AVVI backups @ 4000 MB/min. When I have my new hardware comingo for my lab I'll test it.

Thanks for your help!

Acciaioli's picture

Hello,

I'm experiecing the same issue after I've enabled the GRT option. Before I had no issues with the average speed of my Exchange backups.

In our specific case, we use the RAWS agent instead of the AVVI, and the Exchange running on a VM.

If I revert the GRT option like it was before (disabled), I'm almost sure that will solve the backup speed issue. However I wanted to have this option enabled because I might need to restore an individual mailbox or public folder.

Your thoughts are highly appreciated!

Thanks,

Acciaioli

Marco Tafur - Qualitas Technology's picture

I haven't found any answer to it yet. My customer is just happy with AVVI. So, I encourage you to use the same setup.

Howeber, be advised if you have MBOX and CAS roles in different servers, you will have to redirect exchange mailboxes / objets restores to the CAS server in order to have a successfull job.

Exchange with no GRT is like backing up files, and you won't have the flexibility to restore some objects. Have you opened a support ticket with Symantec? 

Marquito's picture

Hi,

Sorry for the delay (my previous post was done with my colleague's account), but in the meantime I've opened a case at Symantec hotline. We went through an update to SP1a, updated the backupexec agent from Exchange, and the result was exactly the same. I also tried to backup directly to the tape I didn't faced slowing issues. As a matter of fact, I was suprised with the job rate, which has done with an average speed of 3.500 MB/min!!! However, I've told Symantec support team that the idea it's to backup to disc and then duplicate to the tape.

They've collected logs from both backupexec 2012 server and from the exchange backup agent as well, and in the meantime they've escalated to their engineering team, since this might be a kind of bug.

I will keep you posted!

Regards,

Marco

JuergenB's picture

Hi,

i had a similar problem with a GRT Restore of some mailbox items.

The staging was done with 200MB/s  to C:\Temp and it was slow.
The Storage is a local Dell T110 Server with RAID1 set.

I moved the GRT staging folder from the local drives to a Dell MD3000 iSCSI Storage.

The staging is done now with 970MB/s

I use Windows 2008 R2 and BE 2012 SP1a, 3 hotfixes

hope this help you to find the bottleneck.

ub40's picture

Hi, all !

I'm confirm, bottleneck is GRT mode Exchange Backup.
In this mode after every base backup, server rebuilds individual messages catalog for restore separate mail items facilities.
The greater the number of mailboxes and messages in the base, the slower backup.
For example : after enabling GRT backup rate base about 200 mailboxes is halved.

Regards,
ub40

derekmines's picture

I have the exact same issue as this.

Using Backup Exec 2012 to backup an Exchange 2010 server onto an HP Ultrium tape drive and I'm getting speeds of 60Mb / min

Unfortunately all the talk above of GRT mode, AVVI, MBOX, CAS etc has completely lost me.

Could some kind soul please give me a step by step to sorting this problem?

be-nugget's picture

Also ensure that beremote.exe is added as a process exclusions in your AV.

Nabil polska's picture

check your anti virus  if he is scanning files/mailbox etc during the backup , with many AV it's checked by default