Data Doubling Every Two Years

Enterprise investment in the digital universe—cloud, hardware, software, services, and staff to create, manage, store and generate revenue from the information—has increased 50%, to $4 trillion, since 2005, according to an EMC-sponsored IDC Digital Universe study. The forces behind this relentless growth are driven by technology and money. New "information taming" technologies are driving the cost of creating, capturing, managing and storing information down to one-sixth of what it was in 2005.

According to the study, "Extracting Value from Chaos," the growth isn’t ceding anytime soon. The world's data is doubling every two years—growing faster than Moore's Law. IDC, which conducted the study forecasts 1.8 zettabytes (1.8 trillion gigabytes) will be created and replicated in 2011, and the Digital Universe and Big Data are driving transformational societal, technological, scientific and economic changes.

IDC notes that the skills, experience and resources to manage the deluge of data and resources simply isn't keeping pace with all areas of growth. Over the next decade, IT departments worldwide will experience:
• 10 times the number of servers (virtual and physical).
• 50 times the amount of information to be managed.
• 75 times the number of files or containers that encapsulate the information in the digital universe, which is growing even faster than the information itself as more and more embedded systems, such as sensors in clothing, in bridges, or medical devices.
• 1.5 times the number of IT professionals available to manage it all.

While cloud computing accounts for less than 2% of IT spending today, IDC estimates that by 2015 nearly 20% of the information will be "touched" by cloud computing service providers — meaning that somewhere in a byte's journey from originator to disposal it will be stored or processed in a cloud. Perhaps as much as 10% will be maintained in a cloud. 

"The chaotic volume of information that continues growing relentlessly presents an endless amount of opportunity—driving transformational societal, technological, scientific, and economic changes," says Jeremy Burton, Chief Marketing Officer, EMC Corporation. "Big Data is forcing change in the way businesses manage and extract value from their most important asset – information.

So what can IT do? The study suggests that new capture, search, discovery, and analysis tools can help organizations gain insights from their unstructured data, which accounts for more than 90% of the digital universe. These tools can create data about data automatically, much like facial recognition routines that help tag Facebook photos. Data about data, or metadata, is growing twice as fast as the digital universe as a whole.

Other considerations from the study include:
· Business intelligence tools are used for real-time data, whether it's charging auto insurance premiums based on where people drive, routing power through the intelligent grid, or changing marketing messages on the fly based on social networking responses.
· Storage management tools cut the costs of the part of the digital universe we store, such as de-duplication, auto-tiering and virtualization, as well as to help us decide what exactly to store, as in content management solutions.
· New security practices and tools can help enterprises identify the information that needs to be secured and at what level of security, and then do so from specific threat protection devices and software to fraud management systems and reputation protection services.
· Cloud computing solutions—both public and private and a combination of the two known as hybrid—provide enterprises with new levels of economies of scale, agility, and flexibility, compared to traditional IT environments.

CIOs and their staff may be able to manage these new tools, but organizations have to be prepped and ready to deal with the nuggets that can now be pulled out of the digital universe.

The study highlights a call to action that has two components—one technical, one organizational. Technical calls to action include investigating the new tools for creating metadata, mastering virtualization, moving what you can to the cloud, determining which Big Data projects will have the most "bang," staying close to the latest information security strategies and practices and being aggressive in developing and managing advanced storage management tools. Organization calls to action include setting the strategy and build, with other C-level executives, a process for migrating to shared resources, beginning to lay the organizational groundwork today with the specific analytical and managerial skill sets, mindsets, and processes necessary to extract the most value possible from Big Data, working to make your company a data-driven organization and pressing suppliers and business partners to help.

For a full copy of the study, click here.

 

For reprint and licensing requests for this article, click here.
Analytics Policy adminstration Data and information management Compliance Security risk Core systems
MORE FROM DIGITAL INSURANCE