Allstate’s Claim Prediction Challenge Nets Insights

The quest for predictive modeling insights has proved a boon for Allstate, as it releases the results of its three-month long crowdsourcing competition, which set out to find the best models predicting bodily injury claims based on vehicle characteristics.

Crowdsourcing is a distributed problem-solving and production model in which problems and or data are broadcast to an unknown group of solvers in the form of an open call for solutions. Users—also known as the crowd—typically form into online communities—in this case using the Kaggle platform, and the crowd is encouraged to submit solutions. The best solutions are then owned by the entity that broadcast the problem in the first place—the crowdsourcer, in this case Allstate.

Allstate launched the "Claim Prediction Challenge" on Kaggle on July 13, offering $10,000 to number crunchers worldwide. Even as of October 12, contestants continued to submit algorithms down to the competition's closing minutes.

From 1290 total submissions and 202 players, three data scientists rose to the top of the predictive modeling competition:

1st place - $6,000 – Matthew Carle – Sydney, Australia

2nd place - $3,000 – Owen Zhang – Bolton, Conn., USA

3rd place - $1,000 – Jason Tigg – London, United Kingdom

The three winners said the competition was about more than monetary rewards; they entered to size up their skills against some of the best predictive modeling talents in the world. The public leader board on the competition site fueled continual model improvements and additional entries.

"As an actuary, I have worked on claims models in the past, and the Claim Prediction Challenge allowed me to see how my modeling skills compare with those of other modeling experts," said Carle. "It also provided a way to improve modeling skills and try new techniques."

Allstate says it provided data from 2005 to 2007 to contestants, who analyzed correlations between vehicle characteristics and bodily injury claims payments to predict claims payment amounts in 2009. Using the data provided, modelers evaluated how factors such as horsepower, length and number of cylinders affect an insured's likelihood of being held responsible for injuring someone in a car accident. Allstate confirmed that the data provided to contestants contained no personal information about any individual consumers.

"A competition of this type is definitely a new approach for the property and casualty insurance industry, and we're proud to reward the work of these talented winners," said Allstate VP Eric Huls, Quantitative Research and Analytics. "Allstate always looks for ways to embrace new ideas and appreciates the excitement and participation generated in this community of modelers."

Allstate plans to examine the winning modelers' methods for possible use with its existing best-in-class modeling techniques, the insurer said.

"We're excited to see Allstate helping to advance actuarial science by engaging the best and brightest data scientists from around the world," said Anthony Goldbloom, founder and chief executive officer, Kaggle. "Kaggle's competitive dynamic inherently pushes entrants to produce better work than they would have if they were working in isolation, so they break the benchmark time and time again. This ultimately leads to greater efficiency for companies like Allstate."