Where insurance professionals lose trust in AI: SAS

Insurers are deploying AI across the business, but adoption brings a new set of risks alongside the promise of greater efficiency. As AI systems become more autonomous, insurers must balance speed and scale with accuracy, oversight and accountability.

Processing Content

According to a study on employee trust in AI from SAS and IDC, the top reason an employee overrides AI in the field is when the technology cannot provide an explanation for its decision.

SAS defines trustworthiness as a reflection of the practices, controls and governance mechanisms that make AI worthy of trust. According to the SAS study and research insights by IDC, organizations across industries that invest in trustworthy AI consistently outperform those that do not. The study found that the insurance sector sees this most clearly among the industries studied: 59% of trustworthy AI leaders report strong or high AI ROI, and AI laggards report no clear ROI at all.

The research measured five elements of trustworthy AI in insurance on a 100-point scale: data quality and governance, 60%, model governance and oversight, 60%, explainability and fairness, 62%, responsible AI policy, 70%, and audit and accountability, 64%. According to SAS researchers, insurers have already laid the foundation for governance and policy. The report suggests sustaining that pace and AI deployment scales in order to keep tech capabilities and consumer confidence high.

"When AI works, it's incredibly impactful. However, it is well documented that state-of-the-art agents can have error rates that exceed 25% on complex tasks — which is unacceptable in high-stakes decision-making," said Bryan Harris, CTO at SAS. "In order to achieve accuracy and repeatability, organizations must embed domain expertise into agentic workflows, while keeping people at the center of governance and oversight. Organizations that do this successfully will close the trust gap and gain a competitive advantage in the market with AI."

SAS data shows that because risk management is fundamental to the insurance industry, insurers that established strong AI governance early are now better positioned to turn AI investment into measurable returns. 

When it comes to agentic AI, 34% of leaders cite managing autonomy and human oversight as a concern, versus 21% of laggards. This higher level of concern may be due to how AI leaders are already deploying more autonomous systems and confronting governance questions that come with them.

"As AI becomes more autonomous, organizations face a new challenge: maintaining confidence in systems people don't fully understand," said Chris Marshall, vice president at IDC. "Our findings show that stronger oversight, explainability, accountability and data foundations are becoming prerequisites for scaling AI successfully."


For reprint and licensing requests for this article, click here.
Agents Automation
MORE FROM DIGITAL INSURANCE
Load More