With the amount of talk about big data recently, you'd be forgiven for thinking that it was all a done deal. The truth is, however, that the can of worms has only just been opened. What started as a reasonably succinct view of the distributed information structures required to support highly scalable, event-driven online sites (think: Hadoop) has degenerated into a general discussion about the data explosion and what to do with all the islands of information that exist out there.
While purists might complain with the disintegration of the boundaries around big data, this may be no such bad thing. The fact is that companies large and small are struggling with the burgeoning size of all their data repositories, and who would not want to be able to use all available information as a business asset?
The quantities of information organizations are dealing with today bring new challenges, from 'simply' being able to shift terabytes of data from one place to another, to ensuring that the right people have access to the right resources - and being able to monitor who does what, with what. Fair to say that these are not new problems, but what has changed is the increased customer priority to get a bird's eye view of all data sources, where they are and what they contain, so that they can be fully exploited.
It will come as no surprise to anyone that we have solutions in this area - data management and navigation are part of Symantec's DNA, and indeed only last month we launched the latest version of Symantec Data Insight, which enables decision makers to effectively manage the huge pools of unstructured data that organizations now have to manage. Go us - but effective exploitation of big data is going to require more than a simple technology solution, however well-architected it may be.
Alongside the challenges relating to data growth and complexity run huge opportunities for those organizations that can derive business-changing insights from their own Aladdin's caves of information. Finding such answers may be like looking for needles in haystacks, but in this world of compressed margins and efficiency savings it will be insight, not information that means power.