Video Screencast Help
Symantec to Separate Into Two Focused, Industry-Leading Technology Companies. Learn more.

EV Move Archive - speed / transfer rate example

Created: 30 Apr 2013 • Updated: 06 Jul 2013 | 9 comments
cl12's picture
This issue has been solved. See solution.

Hi Guys

has anyone got experience in doing a move archive over a large number (100+) of archives and can give me an idea of the time required in terms of items / MB per second / hour etc that Move Archive can process the data ?

i know that the underlying storage / processor will have a bearing but any examples you have will be great then i can average it out

thanks

Operating Systems:

Comments 9 CommentsJump to latest comment

TonySterling's picture

Have you seen the Move Archive section of the perf. guide?

Enterprise Vault 10.0 - Performance Guide

Enterprise Vault 10.0 - Performance Guide
DOC4553 | 2013-01-18
Documentation | CMS-XML

It covers off some of the caveats and is worth a look.

SOLUTION
Rob.Wilcox's picture

Remember that there are a huge number of variables involved ...  so the best bet is to test it on1-2 archives in your environment, and then estimate from that.

MMcCr's picture

I have used MA in anger to move large numbers previously, and as Rob correctly points out, you need to test it on a couple of samples of 10-20 first after first making sure it works fine, there are so many variables at play one move will not be the same as anouther. 

I did the bulk of my moves using EV8sp5, EV9+ is a little better at managing the proccessess available during the different move archive phases - however it is still good to get to know how you can increase the threads of Move Archive using the config file and test on your system to make sure you're not over loading things (i.e. how many threads dedicated to the copy etc).

When doing my moves I got figures simliar to the following: c.1.5gb/hr in the same datacenter, Centera storage, c.1gb/hr between datacenters off a poor source server (but archiving had been stopped and minimal access to data during the moves - still maxed out the source server on a few occasions).  On average the number of archives moved perday where 15-22(in same DC), 10-15(remote DC).  Al done in a window avoiding the backup window of 1am-6am.

One key thing that will trip up any large scale moves with MA (apart from the fact that it wasnt designed to do it and you'll struggle to find anyone with much experiance of using it for such as its for ad-hoc moves of arround 10 or so at a time) is the proccessess and the phases, if you're moving from 1 EV to 1 EV you will really have to ensure that you're copy phase threads arent being used up by verification phase, really test that as in EV8sp5 you had two proccesses that could run at anyone time from one source server, wether that be phase1, 2,3 or 4 etc.  If moving from multiple sources to multiple targets you get much better performance as long as you spread your archives to a couple per server.

Key thing is to test it thouroughly before commiting to any management team or project that you can do x-per day - make sure you can hit that first.

 

 

 

 

Chris Warren's picture

Expanding a bit on MMcCr, if you are planning on performing a large number of MA's, please be sure you are at 9.0.4 or above.  There were a number of issues with recombining legacy items which were fixed in 9.0.2 and 9.0.3.  

As for the working through Moves, I agree with everyone on testing with 1 or 2 archives first.  Making sure they Complete before you lock yourself into moving 100+ archives. While I understand that the original concept of MA was not designed for major migrations, I have seen my share of companies using it as their sole migration tool for 1000's of archives across numerous sites, since it was free.  

Commonly, I've seen, if it were to fail, the most common time for failure has been during the Copy Stage (Stage 1).  During this process, Storage is extracting all the items for the archive.  If items fail to extract or recombine, that can fail it right there.

This document may also give you more details on the process:

http://www.symantec.com/docs/TECH129236
Enterprise Vault Move Archive Feature Overview
 

Hope this helps.

TheEmptyMind's picture

We recently moved around 1K archives from one site to another.  We found during testing that Move Archive was way slower than what we anticipated with plenty of resources not utilized.  Increasing the number of threads did not have noticeable impact.  Also, we did see failures updating shortcuts for a few archives which didn't seem very promising.  I wish I had empirical stats for you, but sorry, I couldn't locate my notes.

We opted for a third-party solution called Migrate (from www.globanet.com).  With it, we are able to migrate over 35-40K items/hour (over WAN) with the source CPU hovering between 30-55% CPU.  The nice thing we like with the Migrate tool is that we can resume the task should something fail during the move process for e.g., if the EV services were restarted during the move, and the tool starts where it left off.

Note that I do not work for Globanet :=)

Rob.Wilcox's picture

FWIW we've got some lightning fast performance recently out of Archive Shuttle when moving archives. Extracting at 100 GB per hour...

http://ow.ly/i/1UGzs

And the full dashboard graphic showing ingest speed as well:

http://ow.ly/i/23jhh