Part two of a conversation with Jodie Wallis, the first global chief AI officer for Manulife. Wallis spoke with Digital Insurance about responsible AI and how the insurer is deploying the technology. Wallis was previously the global chief analytics officer. Manulife recently released
This conversation has been lightly edited for clarity.
Editor's note:
Can you share how you consider sustainability when deploying AI?
AI depends a lot on computer resources, which depend on large data centers, which are huge power consumers and growing.
AI uses computer resources in the training of the models with Gen AI. We don't train our models. We use foundation models like Open AI in our solutions, and then it's also on the execution of the model. So there's power usage in both. On the first one, we are looking to partner with organizations and use foundation models from organizations that share a commitment to sustainability, that have also stated principles around making sure that they're being responsible in terms of power consumption.
In terms of the execution side, which we do control. It's things like I mentioned earlier, we're not just picking the biggest and the latest model when we're designing solutions, we're looking for the most efficient model that meets our accuracy thresholds. So that is a discipline that is in the design process that we use for every new solution. It's very easy to get excited about a new model and say, 'Oh, I want to use that one, that sounds fantastic, and then put it into a use case. It doesn't really need it.' So, finding the right intersection between enough power to get to the accuracy that we need, but not more than enough so that we're really minimizing our footprint.
What have you noticed about the usage of AI?
I have been very surprised at how many people want to, and are trying to incorporate it into their day to day. So we made a decision in late 2023 or middle of 2023 to create an internal version of a general purpose Gen AI tool, which we call Chat MFC. And we decided that that was going to be rolled out to 100% of our colleagues and our contractors. And so we built that in 2023, piloted it, and then rolled it out in earnest to everybody in early 2024 and we put some adoption resources behind it. What I mean by that is people who were available to help if you needed help, or to show you how to use it in your own context. We built this program called Promptathons, where we get a bunch of employees who have the same job to bring their laptops to a session, a virtual session, and then they actually do hands-on practice of prompting for processes and work that's meaningful to them. And we have achieved a 75% utilization rate across our 40,000 users, to me, that is tremendous.
Any advice for other insurance professionals related to this topic?
Don't underestimate change in adoption resources. I think we sometimes, as large organizations, tend to put all of our money into let's build the tech. And here it is really important, because it's a different way of thinking, right? It's a different way of working for our colleagues. So I think don't underestimate the change in adoption that's required.
I think the second thing is, really to find business champions. Like we as AI professionals, can't champion this alone. No matter how good we are. We will not scale these solutions across the organization unless we have really great business champions, and they exist in, I believe, truly, they exist in all organizations. So it's a question of finding them, partnering with them, and kind of leaning in with them. Also, maybe the difference between deterministic and probabilistic solutions is large, and I think it's underestimated. So I've seen companies go in and use kind of existing methodologies and testing processes and say, 'Okay, I'm going to go big on Gen AI,' and they can't quite get to production because they'll never meet the thresholds that are required for accuracy and transparency, explainability, things like that. So it's really recognizing that there is a different methodology, a different skill set, a different way of thinking required for the producers of these solutions and the consumers of these solutions, and very quickly adapting existing processes to accommodate for that.
I would also say bias test. I think bias is the opposite of fairness. So fairness is what we're trying to achieve, and bias would be the negative outcome. So I think of fairness testing as one of multiple dimensions that we need to kind of equally pay attention to. So we have fairness accuracy, I've mentioned a couple of times, certainly explainability is very important, particularly in some use cases. And then there's appropriateness, right? We're asking Gen AI to generate content for us. So appropriateness, which is a bit amorphous, but very important. And then sustainability or efficiency, depending on how you want to look at it. So, we take all of these factors in and like most large organizations, have a model risk management process. And we have a very specific sub-process within that for learning models, for machine learning models and AI models. And we start by assessing all of our use cases in terms of their materiality, and then depending on the materiality, the models are then reviewed by an independent group, either within or even outside of our organization, depending on the severity. And I think that's a process that most organizations use pretty consistently. But I would say that what's more important than the bias testing and the accuracy testing is actually fairness by design and accuracy by design. So not waiting until we build a solution to then test and say, 'Hey, how did we do on these factors?' But really shifting it left quite significantly to say, 'How do we design something that has fairness embedded, that has appropriateness embedded.' What are we doing in the selection of the data, in the selection of the model, in the writing of the code to build it in from the beginning, I think that is a practice that we are striving to continuously improve and evolve.
