Video Screencast Help

Recovery Solution Agent Performance Issues

Created: 21 Apr 2009 • Updated: 21 May 2010 | 10 comments

Hello Everyone,

I was curious to know if anyone has seen or knows of ways to improve recovery solution agent performance. We do partial snapshots of certain directories at our branches and normally we are okay for the first partial snapshot taking 3-15 hours depending on how much data is there. However, with some of our machines, we are noticing them doing the partial snapshot for the first time and requesting 3-22 days worth of time to perform the snapshot. Obviously this is a little rediculous. The change rate is slim, in fact most of the time their future backups only take 3-8 minutes and its not an issue with speed of connection to the locations. There is no more than 2-3 gigs worth of unique data per machine and we're currently running 15-30 jobs at a time for the upwards list of 600 clients. The frustrating part is we cannot get them completed in time before the old snapshots of other systems get deleted which just makes it frustrating to keep up with.

Sadly we are seeing some of these machines taking forever when they have 2 core CPUs and at least a gig of physical ram running XP.

If there are certain processes, whether system, altiris, etc. that need to be excluded from any scanning, or if theres any other things that i can check on that would cause it to take forever, let me know. Again, this is not an FSR, and if it would improve the speed by removing any OFM we would do that as well (This is more out of convenience than a true recovery solution).

If you have any experience in what we SHOULD look into, let me know- the performance of the actual server itself is fine.

Comments 10 CommentsJump to latest comment

alazanowski's picture

We are Running Altiris RS 6.2 SP3, all latest hotfixes applied.

I'm noticing on some of these that the RS Agent is growing to 500mb worth of memory use. Not sure if this is normal or not; especially when you try to stop the service when nothing is running and it hangs.

We're aiming to make sure that theres nothing in our environment that would cause a slowdown before we hand it over to symantec technical support.

-Austin Lazanowski Backups cost way too much until you needed them.

KSchroeder's picture

What AV vendor do you use?  We have Trend and haven't (yet) run into any issues with it conflicting with the RS Agent.  Depending on the AV product/configuration, it could be spending alot of cycles trying to scan the file/data stream as it is sent up from the client to the RS server.  I take it you don't see these delays when taking initial snapshots at the main office where the RS sits, on an identically configured machine?

Thanks,
Kyle
Symantec Trusted Advisor

For Forum threads, please click "Mark as Solution" if answered.
For all content, please give a thumbs up if you agree with or support the post.

alazanowski's picture

Actually that is a good point. I will have to test one of their machines locally when it comes in durnig replacement and see if its actually improved on the speed.

We currently have McAfee and webroot spysweeper (yes... i know how terrible that is) plus a HIPS application on these machines. I don't necessarily see a performance spike when they are running, and even when i've restarted the service on the machines it still performs at the same speed (whether or not its currently consuming alot of memory).

-Austin Lazanowski Backups cost way too much until you needed them.

hello techs's picture

Its a useful to me know about much more of this from the article, so I am adding to this my most-reads.

KSchroeder's picture

The above links to the RS 6.2 SP1 hotfixes; Outbacker said he has RS 6.2 SP3 and already has the latest hotfixes:
https://kb.altiris.com/article.asp?article=45676&p=1

So to be sure I understand, even with McAfee disabled, the performance is still extremely slow?  What about if the HIPS program is disabled?  Have you tried running FileMon (http://live.sysinternals.com/filemon.exe) on the machine while it is running the snapshot to see if any errors are logging there?  Can you add some sort of exclusions to the HIPS software to not scan the AeXRSAgt.exe process and/or traffic to/from the RS cluster/server IP?

Thanks,
Kyle
Symantec Trusted Advisor

For Forum threads, please click "Mark as Solution" if answered.
For all content, please give a thumbs up if you agree with or support the post.

alazanowski's picture

So what we found is that there is a directory that has Several thousand folders and below that several more thousand files that are small in size (between 1-20kB). The overall directory from the top layer is only 1.5 gb which we are fine with whatever it takes to back all of that up, but apparently the slowdown comes from RS attempting to access each file individually, take a look to see if its in active use or not, do its sql stuff for making sure not to backup data it shouldnt, and then has the wonderful issue of attempting to back that one tiny file up and then go to the next. When we removed this directory (they still have 5gb of other data that is being backed up) the backup speed jumped through the roof and they are finishing within 2-10 hours instead of 22 days.

So i guess there is a significant issue with RS attempting to handle a large amount of directories and files; is there a workaround to this situation? I considered having it nightly send the changes into a zip file but i dont want it to have to backup the entire zip file everyday as a small amount of new files are added.

We tested the same filesystem on a blank installation of XP and it had the issue with that folder, so it ruled out other software being involved.

-Austin Lazanowski Backups cost way too much until you needed them.

KSchroeder's picture

Hi Outbacker,
I'll see if support can take a look at this one.

Thanks,
Kyle
Symantec Trusted Advisor

For Forum threads, please click "Mark as Solution" if answered.
For all content, please give a thumbs up if you agree with or support the post.

alazanowski's picture

I have a case opened and sent over logs. They had me run this diagwin product to log everything while the OFM logging was also enabled. Haven't heard back from the developers yet. Unfortunately i think if they cant find a quick solution that we'll have to start considering other products. Its unfortunate since RS has the potential to become a very useful product if a few minor adjustments could be made.

-Austin Lazanowski Backups cost way too much until you needed them.