Video Screencast Help

Critical System Protection Log Storage

Created: 13 Jun 2013 • Updated: 14 Jun 2013 | 3 comments
This issue has been solved. See solution.

We are in the process of trying to estimate the size of the file share storage that we will be storing our bulk logs onto.  We are unsure about what size of storage or disk space is needed to store our bulk logs.  We are planning to deploy agents to 10,500 devices.  The logs will be rolled up every 24 hours and brought back to the server.  We are planning to keep a 3 months of current logs on the server and 10 months onto tape for PCI compliance and investigative purposes.  We need help in trying to find out the size of the storage.  Any help would be appreciated.

Comments 3 CommentsJump to latest comment

Alex_CST's picture

So its PCI compliance that you're implementing?  Is it just File Integrity Monitoring that you need to do?  It's such a difficult question as there are literally thousands of variables that affect the file sizes.  Has a POC been done?  If you're not into the POC stage yet, I would suggest doing one.  

You can then calculate the estimated storage requirements for 10,500 endpoints if you do a POC for 30-40 devices and do the maths, that is going to be the most realistic way of calculating in my estimation.

Feel free to send me a privats message if you want to discuss it in more detail

Please mark posts as solutions if they solve your problem!

AMoss's picture

The 'bulk log' functionality is unique, and has it purposes, but it is generally not something that I recommend folks leverage...ESPECIALLY for compliance.

Now...if you're just talking about regular CSP events, those are transmitted and stored directly in the SQL db.  An example is below re: SQL storage.

If you're deploying to 10k devices and you're new to CSP, I would recommend engaging a consulting group...even if it's only for a week to get the low down on best practices.

For some VERY rough math...without having access to more info about your environment:

Each event consumes ~2k.  An average machine with policies tuned for PCI (NOT including succesful login data) should be ~20 events per day(this number can vary WILDLY). So if you are retaining 90 days nearline..that means each agent will have ~1800 events consuming ~3.6MB  Multiply that by 10,500 and you're looking at about 38 GB.  Realisticly I would NEVER size a db for 10.5k agents at 38 GB as you leave yourself very little margin for error AND you're assuming that your policies are very well tuned.  My rough estimate for a 10.5k machine environment would be minimum of 200GB SQL space.

Hope this helps and feel free to ping me if you need additional guidance.

Looking for real-time reporting and data visualization for your Symantec Security solutions?

Want to tune DCS/CSP like a master? Cut your tuning times by more than half!

Tim_B's picture

Thank you all for the replies.  We got word back from our vendor that we did our POC with and they are saying that we will be averaging about .1 event each second so we figured it out to 20,000 events a day per device so we feel comfortable sizing our file share to be a minimum of 1TB for a year of storage.

Thanks again to everyone.