Bias in AI 'is not unexpected': Varun Mandalapu at Mutual of Omaha

August 16, 2023 6:11 AM

What can data scientists do to ensure the companies they work for use artificial intelligence responsibly?

Digital Insurance received written responses from Varun Mandalapu, senior data scientist at Mutual of Omaha, about how the role of data scientist has evolved within the insurance industry. He also shares some best practices for effective collaboration between data scientists, engineers, data analysts and business stakeholders.

Mandalapu earned a Ph.D. in Information Systems from the University of Maryland, where he specialized in AI and knowledge management.

At Mutual of Omaha, Mandalapu is responsible for leading the development of innovative data science solutions. He also has led the development of tools for identifying bias in AI models.

How has the role of data scientist evolved in the insurance industry over the last few years? How do you see the role continuing to evolve?

The role of data scientist in the insurance industry has experienced a significant transformation. Previously, data scientists primarily focused on data collection and analysis, particularly consolidating data from legacy systems. The main goal at the time was to understand the data and its characteristics, and there were limited use cases for advanced analytics as business was also evolving in this space. With the advent of big data, companies started focusing on enabling data for analytics and predictive modeling. Data scientists now play a vital role as crucial collaborators with business stakeholders by utilizing predictive modeling, machine learning algorithms and data-driven decision-making. They work closely with insurance underwriters, actuaries, claims professionals and marketing teams to identify new growth opportunities, develop personalized insurance products, and optimize pricing strategies based on data insights.

As the insurance industry embraces data-driven approaches, there is a growing emphasis on the fair, transparent and interpretable use of AI and machine learning models. As a result, data scientists are also collaborating with privacy and risk management teams to implement best practices and ensure insurance companies meet regulatory requirements and protect fairness.

As the industry becomes more data-driven, data scientists will continue to evolve and be at the forefront of innovation, spearheading initiatives to harness the full potential of data for improved customer experiences, efficient operations, fraud detection and strategic decision-making. The integration of AI and automation will streamline processes, enabling faster and more informed real-time analytics for instant decision-making.

Can you elaborate on your work in identifying demographic bias in machine learning models?

As the adoption of sophisticated machine learning algorithms increases, the potential for bias in these models becomes a critical concern. From a data scientist's perspective, bias is not entirely unexpected as models capture the patterns present in underlying data. However, the significant shift came when data scientists started realizing its real-world impact.

In my work, I focus specifically on studying bias during the model development phase. I have been actively working on building statistical indicators that quantify the level of bias in machine learning models based on sensitive demographic information. This involves investigating how model predictions vary between different demographic groups, seeking to understand if there is any presence of bias and, if so, which group the model may favor.

We are also working to develop methods to quantify bias even when sensitive demographic information is not directly used in model development. For this purpose, we investigate whether there are any hidden relationships between the features models are trained on and the sensitive demographics, and how much impact these features have on the model's predictions.

To ensure the reliability of our models, we have also adopted a model review management process traditionally used for actuarial model evaluation. In this process, a data scientist from a different team assesses the model built by another team to evaluate if there are any associated risks, including bias. My team is currently working on standardizing a template that integrates explainable AI methods and fairness evaluation metrics into our model-building pipelines. This integration ensures that evaluation for bias and interpretability takes place while data scientists work on building models. By incorporating these measures early in the development process, we can proactively address any potential bias issues and promote fair and transparent model outcomes.

What role does explainable AI play in the development of responsible and ethical AI models, and how are data scientists addressing the need for interpretability? What strategies are data scientists adopting to monitor and mitigate bias in models, and ensure ethical AI practices?

Explainable AI is a topic that is particularly close to my heart, and it was also one of the research areas I focused on during my doctorate. In my view, the ability to interpret and explain AI models is of utmost importance, especially as these models become increasingly integrated into complex decision-making mechanisms. In the development of responsible and ethical AI models, it enables us to understand how AI models arrive at specific decisions and predictions, providing crucial insights into the reasoning behind their outcomes. This transparency is essential, especially when AI is employed in high-stakes applications like underwriting, pricing and product recommendation systems. By having a clear understanding of the model's decision-making process, we can identify potential biases and make necessary adjustments to ensure fairness and accountability.

Data scientists are actively addressing the need for interpretability by exploring various explainable AI techniques. Some of these techniques include feature importance analysis and model-agnostic methods, such as LIME (Local Interpretable Model-agnostic Explanations) and SHAP (SHapley Additive exPlanations). By leveraging these methods, we can gain deeper insights into the factors driving model decisions and present understandable explanations to end users and stakeholders.

In terms of monitoring and mitigating bias in models, data scientists are adopting a multi-faceted approach. We begin by meticulously evaluating the data used to train the models, identifying potential biases present in the data collection process and understanding the representation in data based on sensitive demographics. Strategies such as fairness-aware data preprocessing, bias-aware model selection and adversarial testing can be employed to detect and address bias during the model development phase. Ongoing monitoring of AI models in real-world scenarios is also vital to ensure that biases do not manifest or amplify over time. Regular audits and performance evaluations help data scientists proactively identify and rectify any emerging bias issues.

To ensure ethical AI practices, data scientists adhere to established guidelines and industry standards. We prioritize ethical considerations during every stage of model development and collaborate closely with domain experts and legal professionals. Additionally, the adoption of model review management, where models are evaluated by cross-functional teams, plays a significant role in validating the ethical soundness of the AI models.

How do you envision the future of AI regulation in the insurance industry?

I believe that AI regulation in the insurance industry is likely to become more stringent in the future. As we gain a deeper understanding of how biases can impact real-world outcomes, changes in AI regulations are becoming a priority, especially in an industry like ours, where financial security and fairness are paramount. However, while regulations are necessary to ensure accountability and fairness, we must also be cautious not to stifle innovation in the process.

Finding the right balance between innovation and regulation is crucial, in my view. It requires a focus on continuous research to better identify and mitigate biases in AI systems while simultaneously working toward making it easier for regulatory bodies to make informed decisions about AI applications. The current increased adoption of AI in the insurance sector also presents an opportunity for collaboration between the industry and regulators. Together, we can explore ways to sustainably advance technology adoption, without severely limiting its potential. This collective effort will ensure that technology adoption continues to benefit both businesses and customers while maintaining the necessary safeguards for consumer protection and fairness.

What best practices have you identified for effective collaboration between data scientists, engineers, data analysts, and business stakeholders? How do you navigate potential challenges in this process?

Cross-collaboration teams are vital for successful AI projects, and it is a practice I highly value. By integrating data scientists, machine learning engineers and analysts into a cohesive team, we foster effective collaboration throughout the entire project lifecycle, from defining use cases to delivering models. This approach ensures that everyone involved stays in sync with project objectives. It also streamlines workflows and enables us to find innovative solutions through diverse perspectives. We prioritize clear and concise communication, leverage agile methodologies, and hold regular meetings to track progress and address evolving needs, enabling us to adapt quickly and deliver work efficiently. I also believe that business stakeholder education on what to expect from data science is critical for the success of AI projects. Ensuring that key stakeholders understand the capabilities and limitations of data science models and technologies empowers them to make informed decisions, set realistic expectations and provide valuable input throughout the project.

There can be challenges that arise as a result of different technical backgrounds and competing priorities among team members and stakeholders. To overcome these challenges, we focus on fostering a collaborative culture built on trust, transparency, and data-driven decision-making, which forms a strong foundation for delivering impactful AI-driven solutions. Our team emphasizes the use of common language and work reviews to bridge gaps in understanding. We also employ a data-driven approach to make informed choices in the face of competing priorities. Through effective collaboration, we ensure that our AI projects are aligned with business goals, enabling us to deliver successful outcomes while maximizing the benefits of diverse expertise within the team.

What emerging trends do you see shaping the future of insurance and how is Mutual of Omaha positioning itself to leverage these advancements?

One of the most intriguing trends is the incorporation of dynamic learning algorithms (reinforcement learning), which emphasize continuous learning and updates. These algorithmic methods can be developed using simulation data where availability or access to historical data is lacking. They are deployed to learn and adapt based on real-world changes. While there are infrastructure challenges due to their dependence on external environment responses, I believe these dynamic learning algorithms will see greater acceptance and utilization with the increasing adoption of technology in the insurance industry. Another area I'm focused on is related to generative AI (for instance, ChatGPT). During my doctorate work, I came across research articles that highlighted the underlying functionality and vast potential of this technology. I firmly believe that in the future, generative models will play a more prominent role in the insurance industry, particularly when it comes to enhancing customer experiences. Its integration can revolutionize interactions with customers, providing personalized and efficient services, which has the potential to transform the way we engage with policyholders.

At Mutual of Omaha, we are actively positioning ourselves to leverage these advancements. We recognize the potential of dynamic learning algorithms to enhance our customer acquisition and customer experiences. By investing in the necessary infrastructure and fostering a culture of innovation, we aim to embrace these cutting-edge technologies and stay at the forefront of the evolving insurance landscape. Our commitment to continuous learning and adaptation aligns perfectly with the potential of dynamic algorithms, enabling us to deliver even more personalized and efficient services to our valued customers.

What advice would you give to aspiring data scientists looking to enter the insurance industry?

I would emphasize the importance of grasping the base concepts of data science and understanding the "why" behind what they are learning. Developing a research mindset and the ability to question and think critically will be crucial in navigating the complexities of data science in insurance. Alongside learning data science concepts, I recommend exploring different areas within the insurance industry and brainstorming potential use cases based on your knowledge. Industry white papers provide an excellent starting point. Additionally, it is good to learn about model development in the industry, where the focus is on productionalizing models rather than leaving them confined to notebooks.

As a data scientist, it is also essential to know how to align your modeling efforts with real-world use cases appropriately. Starting with simpler linear methods and gradually moving toward more complex models is a prudent approach. Aspiring data scientists should learn how to identify model biases and work toward making models fair. Prioritizing fairness over pure accuracy is vital, as it directly impacts the customers we serve and ensures ethical and responsible use of data science in the insurance domain.