Controlling the Information Explosion: A Methodology
As discussed in previous blogs the topic of the “information explosion” seems to be top of mind for many IT execs today (unstructured data being the chief protagonist responsible for data growth in most companies). Many IT organisations have, until now, relied on the collapsing cost of storage as a risk mitigation technique & “band-aid” strategy (“well, disks are no longer costing what they used to so we just broker very cheap deals and add racks to the datacentre”)..
This somewhat tactical and short-sighted approach has its obvious short-comings.. For one, 2002 was the last year in which disk price performance kept up with information growth. Secondly, all this “cheap” disk needs somewhere to live (and it needs to be cooled, powered, administered…).
It is predicted that the growth of data will be 800%, with 80% unstructured data in the next 5 years. While enterprise data is growing at a rate of 40% to 60% annually, storage utilization itself often remains at or below 50%.
The question is how do we control information explosion using a comprehensive best practice that can have long-term benefit ?
At Symantec, we have developed a 5 step strategy to addressing this issue. In summary, the approach suggests the following steps:
- Create Storage Visibility. This step involves understanding usage of hardware assets and identifying wasted components, taking control of the storage estate by avoiding over provisioning and simplifying management.
- Create Data Visibility. The step involves a focus on who owns and accesses data, how is the data used and how the data has changed over time . It also deals with techniques to make data loss visible to the point of control and mitigation through security policy.
- Increase Storage Efficiency. This stage relies heavily on de-duplication & archiving technique. Typically, best practice here involves de-duplicating data at source wherever possible, archiving data based on business level policies and automating transparent storage migrations.
- Keep Data Available. This includes both the physical and virtual side of the storage estate (as one, combined asset – google Symantec “V-Ray” offerings for a summary as to how we accomplish this). It focuses on applications, hosts, VMs and sites, protecting the business services, not just IT components and ensures that business critical services take priority.
- Data Discovery. Many organisations have built fairly effective archives with a focus on “storage” (rather the “retrieval”). This step defines the process of identifying relevant data across the enterprise which needs to be collected, centralising the data archive and intelligently managing retrieval and review of content by relevant parties.
Our project experience here has shown the above to be an effective structure for a programme who’s goals are to drive efficiency into the storage environment for long-term business advantage. In our experience, just spending some focused time and effort on “step 1” will, typically, drive 40-60% better utilisation of an existing storage assets..
If unstructured data is spiralling out of control in your business, take a step back, take a deep breath and re-approach the problem using the methodology outline here. You might be surprised at how much value is delivered (and how quickly !).