Video Screencast Help

EVDC - Discovery Collector Performance

Created: 18 Nov 2011 • Updated: 22 Nov 2011 | 12 comments
This issue has been solved. See solution.

All,

 Starting to use EVDC a lot now and after narrowing down its functionality I have a question per below details:

* We are using EV 9.0.2.

* EVDC is a standalone server

* Ran a full harvest for a volume with over 400 GB of data and it took over 2 days.

* We are archiving the 400 GB size harvested data now, but we are getting archive performance rates of about 500 MB an hour.

Is there anything we can do to increase performance besides adding more EVDC VM's and Evault archive servers? What are normal archiving rates per hour when using EVDC and Evault?

 

Please advise,

Thanks for your help!

D

Comments 12 CommentsJump to latest comment

TonySterling's picture

How many CPU and how much Memory for EVDC?

Evault Professional's picture

Tony,

 I believe it has the defaults from the template but we confirmed:

CPU = 4 and RAM = 16GB

Thanks for your help!

D

Dennis
STS Symantec Enterprise Vault 2007, 8.0, 9.0 and 10.0 for Exchange, Netbackup 7.0 for Windows and Symantec Data Loss Prevention 11.0 
SSE/SSE+ Symantec Clearwell eDiscovery Platform and Symantec Data Loss Prevention 11.

TonySterling's picture

How many Harvester processes are you using?

When the system is running how much memory is it using?

Also, is EV doing other archiving when the copy is happening?

Evault Professional's picture

Answers:

How many Harvester processes are you using? The "Harvester" process had already completed when we started the "Copy" process. We do not have any other "Harvester" tasks running with the intent of monitoring accurate statistics during the "Copy" to EV archive. 

When the system is running how much memory is it using? As of now the "Copy" to EV archive process is the only task running and we confirmed that there is about 8 GB space of free memory.

Also, is EV doing other archiving when the copy is happening? EV is currently only perfoming 1 archive process for this single "Copy" of 400+ GB. No other EV archive functionality has been enabled.

Please let me know if you need anything else,

Thanks for your help!

D

 

Dennis
STS Symantec Enterprise Vault 2007, 8.0, 9.0 and 10.0 for Exchange, Netbackup 7.0 for Windows and Symantec Data Loss Prevention 11.0 
SSE/SSE+ Symantec Clearwell eDiscovery Platform and Symantec Data Loss Prevention 11.

TonySterling's picture

I understand you are not harvesting any more but harvester do more than just harvest.  By default a Stand-Alone will have just one. 

For your envionment I would recommend 4 Harvesters as you have 4 CPU's.

SOLUTION
Evault Professional's picture

Just to clarify...Do you mean we should run 4 Harvester tasks on the same data? Or, should we create 4 Harvester tasks on a smaller sets of data?

Example 1:

4 Harvester tasks to run against the same volume data equal to 420 GB size. I'm not sure if this is possible...please confirm if so.

Example 2:

Creating 4 Harvester tasks within the volume data equal to 420 GB in size to archive smaller chunks.

 * 1st harvest based on 105 GB.

* 2nd harvest based on 105 GB.

* 3rd harvest based on 105 GB.

* 4th harvest based on 105 GB.

If example 2; how will that improve archiving rates? Does this mean that EV will archive 500 MB per hour for each "Harvester" tasks listed above even if they will all target a single Evault Server (VStore) at the same time?

Thanks for your help!

D

Dennis
STS Symantec Enterprise Vault 2007, 8.0, 9.0 and 10.0 for Exchange, Netbackup 7.0 for Windows and Symantec Data Loss Prevention 11.0 
SSE/SSE+ Symantec Clearwell eDiscovery Platform and Symantec Data Loss Prevention 11.

TonySterling's picture

Harvesters are not task, they are process threads.  What I mean is to set the number of Harvesters to 4.  Harvesters are what do the work, rather it is indexing the data or copying someplace.  So by increasing the number of Harvesters you increase the number of threads doing work.  You still will just have the one copy job copying mail to the archive, but with more threads doing work.  That means EVDC will be sending more data to EV to archive and hopefully increase your through put.  Although honestly I don't expect it to jump 4 times it should be significant.

Think of it like the Storage Service for EV.  If you set the number of processes to 1 you limit the amount of work it can do to that thread. 

Evault Professional's picture

Thanks for the heads up...

 

 I found the Harvester processes settings. It had been set to 2 and I increased it to 4 per your recommendation. Should I "Stop" the current "Archive/Copy" policy that is executing and restart EVDC so the settings changed can be updated?

 If so; will the job stopped resume once I start if off again? Per EVDC documentation it seems like it will, but want to double check.

 

Thanks again for your help!

D

Dennis
STS Symantec Enterprise Vault 2007, 8.0, 9.0 and 10.0 for Exchange, Netbackup 7.0 for Windows and Symantec Data Loss Prevention 11.0 
SSE/SSE+ Symantec Clearwell eDiscovery Platform and Symantec Data Loss Prevention 11.

TonySterling's picture

Yes, you have to stop it for the new setting to take effect.  Off top of my head I am pretty sure you need to restart the appliance.

And yes, when you resume the job it will resume from where it left off.

Evault Professional's picture

Thanks Tony! I was able to figure out how to restart the services via the EVDC application yesterday, and noticed that the performance has slightly increased. I will mark your response as a solution. However we are still experiencing slow performance as this task is estimating to complete in over 20 days.

I've opened a TKT with Symantec as well in case any events need to be gathered.

Something ODD is that before restarting the service yesterday this tasks was scheduled to complete in 35 days and it was 10% complete. However after the restart it started from 0% again, and it is now at 5%. Why did this not start at 10% again? I'm thinking it excluded the 10% since it had been completed already, and restarted the completed percentage count based on what was still left over. Is that correct?

Thanks again,

D

Dennis
STS Symantec Enterprise Vault 2007, 8.0, 9.0 and 10.0 for Exchange, Netbackup 7.0 for Windows and Symantec Data Loss Prevention 11.0 
SSE/SSE+ Symantec Clearwell eDiscovery Platform and Symantec Data Loss Prevention 11.

TonySterling's picture

Yes, it doesn't take in consideration what it did in the past and will base it's calculation off what is left to process.

Evault Professional's picture

Thanks again!

Dennis
STS Symantec Enterprise Vault 2007, 8.0, 9.0 and 10.0 for Exchange, Netbackup 7.0 for Windows and Symantec Data Loss Prevention 11.0 
SSE/SSE+ Symantec Clearwell eDiscovery Platform and Symantec Data Loss Prevention 11.