All across the land, there is a big-time push to bring in more “data scientists” into companies, to help turn big data into big insights. But who will be assuming these roles, and where will companies find them?

Recent research out of the McKinsey Global Institute finds there is an impending shortage of key talent necessary for organizations to take advantage of big data. Within the next few years, McKinsey predicts the United States alone could face a shortage of 140,000 to 190,000 people with deep analytical skills as well as 1.5 million managers and analysts with the know-how to use the analysis of big data to make effective decisions. McKinsey observes that the insurance and financial sectors are among the sectors that are most likely to be putting big data to work.

Consider a recent ad from Allstate Insurance, in search of a “creative predictive modeler/data scientist to use your machine learning, statistics, data mining, and analytic skills to influence the decision making.” The company sought a professional with a minimum of 3 to 5 years of experience in a predictive modeling/data scientist role in the insurance industry or 4 to 7 years relevant experience outside of insurance.

Where do we find such people? In a new post, Ovum Research's Tony Baer says they are difficult to find. “At last year’s Hadoop World, there was a feeding frenzy for data scientists," he observes.

"Sometimes it seems like we’re looking for Albert Einstein or somebody smarter."

That's because data science is “all about connecting the dots, not as easy as it sounds,” he explains. “The V’s of big data—volume, variety, velocity and value—require someone who discovers insights from data; traditionally, that role was performed by the data miner. But data miners dealt with better-bounded problems and well-bounded (and known) data sets that made the problem more 2-dimensional. The variety of Big Data—in form and in sources—introduces an element of the unknown. Deciphering Big Data requires a mix of investigative savvy, communications skills, creativity/artistry and the ability to think counter-intuitively. And don’t forget it all comes atop a foundation of solid statistical and machine-learning background plus technical knowledge of the tools and programming languages of the trade.”

Again, where do we find such people? Tony points out that while some vendors now offer training, there is a surprising lack of offerings of these skills from systems integrators, and with data science talent scarce, we’d expect that consulting firms would buy up talent that could then be “rented’ to multiple clients. Excluding a few offshore firms, few SIs have yet stepped up to the plate to roll out formal big data practices (the logical place where data scientists would reside), but we expect that to change soon.

The laws of supply and demand will kick in for data scientists, but the ramp up of supply won’t be as quick as that for the more platform-oriented data architect or engineer. Of necessity, that supply of data scientists will have to be augmented by software that automates the interpretation of machine learning, but there’s only so far that you can program creativity and counter-intuitive insight into a machine.

Joe McKendrick is an author, consultant, blogger and frequent INN contributor specializing in information technology.

Readers are encouraged to respond to Joe using the “Add Your Comments” box below. He can also be reached at

This blog was exclusively written for Insurance Networking News. It may not be reposted or reused without permission from Insurance Networking News.

The opinions of bloggers on do not necessarily reflect those of Insurance Networking News.

Register or login for access to this item and much more

All Digital Insurance content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access