If You Think Big Data’s Challenges Are Tough Now…

According to the EMC/IDC Digital Universe Report, data is doubling in size every two years. From 2013 to 2020, the digital universe is projected to grow from 4.4 trillion gigabytes to 44 trillion gigabytes.

For executives faced with the age-old challenge of how to organize and learn from the data available to them, the problem just got a lot harder.

The Internet of Things (IoT) may be a new topic to many executives, but it lies at the heart of the phenomenon of data proliferation. Today, according to the EMC/IDG study, consumers and workers generate two-thirds of all new data. This is about to change. It is projected that within this decade, most new data will be generated not by people — consumers and workers — but by sensors and embedded, intelligent devices connected to the Internet — smart phones, traffic lights, MRI scanners, smart energy grids, and heavy industrial systems.

While still small as a percentage of all data that is generated, it is expected that data generated by “things” will grow from 2% of all data that is captured in 2013 to 10% in 2020, with this pace accelerating during the next decade.

To date, the Big Data challenge for most executives has been about organizing and managing large volumes and greater varieties of historical data, mostly for purposes of improving their analytic capabilities — understanding past behaviors and activity as a predictor of future actions. Historic data is, however, data that’s “at rest”— it teaches us about the past and enables us to forecast about the future, but it is static data.

Big Data, in the context of the IoT, is about data “in motion,” which refers to the velocity and highly interactive nature of the data captured by sensors and intelligent devices. Data is dynamic and exists on a time continuum. As an example, knowing a customer’s preference during a purchase can be significantly more valuable than knowing a customer’s preference after a purchase. The requirements of fresh data imply an ability to ingest, analyze and interact with vast streams of incoming data in real time, requiring an “in-memory” data management approach.

The IoT marries the power of new Big Data approaches for managing and analyzing historical data with high speed monitoring and processing of events as they occur

read more: http://sloanreview.mit.edu/article/if-you-think-big-datas-challenges-are-tough-now/?utm_source=twitter&utm_medium=social&utm_campaign=sm-direct