Video Screencast Help

Archive Rate / Averages?

Created: 20 Jan 2013 • Updated: 05 Feb 2013 | 11 comments
Sarah.Seftel's picture
This issue has been solved. See solution.

Hi

 

any known table that show averages of archiving? ie - how much data can be archived in 1 hour? how much time will it take to archive 100 GB? 

I have a customer that is interested to know what will be the average time for him to archive 100GB.
Although it depends on file size, index, amount of items etc - he wants to know if there is a general archive rate \ average so he can calculate the time it will take him to archive 100GB (more or less) 

Comments 11 CommentsJump to latest comment

Sarah.Seftel's picture

isn't there a tool where we can put the data and get results?
Ho do we calculate the archive rate of 100GB from the tables in the link?

I remember there was a sizing tool before...

Rob.Wilcox's picture

Partners that do Enterprise Vault installations may have a spreadsheet, but, the question 'how long will take to archive 100Gb' depends on a whole range of factors, as you have indicated in your original post.

 

That being said at the top of page 20 in the PDF is an example that says an 8 core server can archive around 60,000 70 Kb items per hour.  That's 4 Gb by my reckoning, so that's 25 hours for your 100 Gb example.

Sarah.Seftel's picture

I'll get the number of cores in the server and calculate it.

 

Thanks!

JesusWept3's picture

First off, sorry for the really really long post

Yeah its just such a difficult thing to calculate, the performance guide is obviously going off of a best case scenario, you may have slightly slower disks on sql/exchange/vault store partitions, you may have network saturation at the point of archiving, you might have exchange maintenance going on etc etc, all of which can slow down the archiving rate

you might come across items that have 10,000 recipients, extremely large items etc, you might have a GC that is slow to respond and it takes time to do security lookups and address lookups etc

You might end up scanning a mailbox with 50,000 folders that has 40GB of mail that takes more time to scan than it does to actually archive anything

The best thing you can do is a pilot against a couple of dozen users, run archiving overnight, or a couple of nights, calculate how many items were archived and the size and then judge it from there

So you could say enable 100 users, run archiving in report mode and look at how many items are eligible for archiving, so it might say that 15,000 items are eligible for archiving, then let the schedule kick in

Then do a query like

 

USE EVVSYourVaultStore_1
SELECT COUNT(S.IdTransaction) "Item Count",
       SUM(S.ItemSize)/1024 "Size After Compression (MB)",
       SUM(SP.OriginalSize)/1024/1024 "Original Exchange Size (MB)"
FROM   Saveset S,
       SavesetProperty SP
WHERE  S.SavesetIdentity = SP.SavesetIdentity
  AND  S.ArchivedDate > '2013-20-01 23:00:00.000'
  AND  S.ArchivedDate < '2013-21-01 03:00:00.000'

 

This will give you the item count and the total size in MB of items stored in EV , and the size of the items gotten from Exchange with a run of 4 hours (11pm to 3am)

So you may get a result like

 

Archived Items Size After Compression (MB) Original Exchange Size (MB)
180,000 5091 5625

 

So based on that you've archived 5,625MB on Exchange and 180,000 items in 4 hours.

To calculate how many items and how much time it took for 1GB we do the following
1. Archived Items / Original Size   (i.e. 180,000 items / 5,625MB) = 32
2. Original Size - 1024 (1GB)  (i.e. 1,242 - 1024GB) = 4,601MB
3. Result of Step 1 * Result of Step 2 (i.e. 32 * 4,601) = 147,232
4. Archived Items - Result of Step 3 (i.e. 180,000 - 147,232) = 32,768 Items in 1GB (1024MB)

So now you know that you have 32,768 items equate to 1GB

Then you need to find out how much time that took
1. Hours to run * 60 * 60 ...(i.e. 4 * 60 * 60) = 14,400 seconds
2. Result of step 1 / Archived Items  (i.e. 14,400 / 180,000) = 0.08
3. Result of Items in 1GB * Result of Step 2 (i.e. 32,768 * 0.08) = 2,621 seconds
4. Result of Step 3 / 60 / 60 (i.e. 2,621 / 60 / 60) = 0.72 (i.e 44 minutes)

So now you have the following results for 1GB of items

 

Size Of Items Time Taken To Archive Number of Items Archived
1,024MB (1GB) 44 minutes 32,768 items

 

Then multiply that all by 100 to get the 100GB
And thats 100GB, 73 hours 20 minutes and 3,276,800 items archived
So if you have a run time of 4 hours per night, that would take 18 nights of archiving to achieve that 100GB goal

 

SOLUTION
John Santana's picture

Wow that's cool to know.

But when I executed the SQL script I got this:

 

Msg 242, Level 16, State 3, Line 2
The conversion of a char data type to a datetime data type resulted in an out-of-range datetime value.

Kind regards,

John Santana
IT Professional

--------------------------------------------------

Please be nice to me as I'm newbie in this forum.

JesusWept3's picture

Sorry, schoolboy error

Change
 AND  S.ArchivedDate > '2013-20-01 23:00:00.000'
  AND  S.ArchivedDate < '2013-21-01 03:00:00.000'

To
 AND  S.ArchivedDate > '2013-01-20 23:00:00.000'
  AND  S.ArchivedDate < '2013-01-21 03:00:00.000'

the issue was that i was doing yyy-dd-mm and it should be yyyy-mm-dd

John Santana's picture

Yes it works for now :-)

Thanks a lot Jesus !

Kind regards,

John Santana
IT Professional

--------------------------------------------------

Please be nice to me as I'm newbie in this forum.

GertjanA's picture

Excellent explanation JW! This is worth an article! Thumbs up from me.

Thank you, Gertjan, MCSE, MCITP,MCTS, SCS, STS
Company: www.t2.nl

www.quadrotech-it.com

www.symantec.com/vision

Rob.Wilcox's picture

.. and a fudge factor in case it's FSA, Domino or SharePoint archiving...