2012 Trend: Big Data
by Hu Yoshida on Dec 27, 2011
The big hype in 2012 will be around Big Data. The explosion of unstructured data and mobile applications will generate a huge opportunity for the creation of business value, competitive advantage, and decision support if this data can be captured, stored, managed, accessed, analyzed, and visualized. Companies that provide these capabilities for Big Data will be targets for acquisition, much like we saw in past years with thin provisioning technology companies.
“Big” is a relative term. What may be big for one company may not be big for another. Data is big when it becomes difficult to work with using relational databases or desktop visualization packages. The size of Big Data makes it impractical to replicate, backup and mine through traditional means. Instead of moving or replicating big data for use by analysis programs, analysis programs will have to work directly with the original data using massively parallel or map reduce software. Big data may be more about the intersection of many data stores.
Big Data is best stored in an object store built on a virtualized storage infrastructure so that it is not affected by changes in the infrastructure. An object store is a container, which holds data along with the meta data that describes the data, and the policies that govern it. An object store disaggregates the data from the application or instrumentation which created it, and essentially virtualizes the data so that it can be accessed and repurposed by other applications.
Since Big Data is predominantly unstructured data that is not updated, once it is ingested into an object store, it only needs to be replicated once for protection. Some data sources maybe acquired over NFS, which will require a high performance filer that can scale to multiple PBs and millions of objects—like the HNAS 3200. HNAS also provides the ability to stub out to the HCP content platform as a seamless extension of the file system into an object store.
The interest in Big Data will drive greater adoption of object stores like HCP and large scale file servers like HNAS from Hitachi Data Systems. It will also open up a huge market for analytics. HDS has partnered with SAP with certification of a reference architecture for their in-memory transaction analysis system, HANA, using our Hitachi Blade servers and AMS storage.
For Hu’s other 2012 trends, visit this bit.ly bundle: http://bitly.com/vXGP2T
[...] about every blogger has decreed that 2012 will be the year of big data analytics, and just in time, we finally get a buh-bye to beta with the Hadoop 1.0 [...]