Almost three-quarters of respondents to a survey by Gartner Inc. said they have or plan to invest in big data technologies in the next 24 months, compared to 64 percent in last year. The number of respondents who said they had no plans for big data investments declined to 24 percent from 31 percent last year.
"Big data investment continues to be led by North America, with 47 percent of organizations reporting investment, up from 37.8 percent in 2013," said Nick Heudecker, research director at Gartner. "All other regions experienced increases in investment over the last year."
According to the research, however, increases in big data investment have resulted in fewer organizations reporting deployed big data projects that anticipated, Gartner said, adding that much of the effort lies currently in strategy development, pilots and experimental projects, as it did last year.
"Last year, we said 2013 was big data's year of experimentation and early deployment," Heudecker said. "So is 2014. In 2013, only 8 percent of organizations reported having big data projects deployed to production. This has increased to 13 percent in 2014, and while still relatively small, represents a sizeable increase. However, the six percent drop in organizations still gathering knowledge about big data and the seven percent increase in pilots and experiments indicate that organizations are evolving in their understanding and willingness to explore big data opportunities."
Lisa Kart, research director at Gartner said big data can enhance the customer experience and improve process efficiencies in an array of industries. "The most dramatic changes are in enhancing customer experience, especially in transportation, healthcare, insurance, media and communications, retail and banking.”
Gartner said the challenges companies face become more practical as they move beyond knowledge gathering and strategy development to making investments, piloting and deploying. In the early stages of big data projects, the top challenges are obtaining skills and capabilities needed, defining strategy, obtaining funding and beginning to think about infrastructure issues. Companies that are further along with investments, Gartner said, must begin to address risk and governance issues, data integration and infrastructure.
In terms of assessing the volume, variety and velocity of big data, Gartner said volume received the most focus. Volume is comparatively easy to manage by increasing storage and compute capacity, Gartner said, Data variety however is more challenging. “Getting value from a variety of data sources, such as social media feeds, machine and sensor data, as well as free-form text, requires not only increased storage capacity, but also different tools and the skills to use them,” Gartner said.
Most organizations are studying traditional data sources for their big data projects, Gartner said, increasing to 79 percent in 2014 from 70 percent in 2013.
"We got a surprising result when we asked respondents which data sources they planned on adding in the future," Heudecker said. "Every data source received roughly 30 percent to 40 percent of responses, including extremely challenging data sources like audio and video. This overly optimistic and apparently random nature of future data sources for analysis indicates two things. First, organizations don't have a plan for what they're going to do next. Picking everything isn't a strategy. It indicates a fear of missing out on an opportunity yet to be defined. Also, there may be a certain amount of hubris at work. If organizations can 'do big data' on transactions and log data, they may assume they can also leverage more challenging data sources as easily."
The Gartner survey included responses from 302 Gartner Research Circle members worldwide, and was conducted in June 2014.
