Building Trustworthy AI in Healthcare with Dr. Peter Bonis, Chief Medical Officer at Wolters Kluwer Health

[00:00:09] Hello everyone and welcome back to the Outcomes Rocket Podcast brought to you by Vive. I'm so excited to be with Dr. Peter Bonis today. He is the Chief Medical Officer at Wolters Kluwer. He's a member of the Executive Leadership Team and oversees content, informatics, and industry partnerships. He's a physician entrepreneur, executive, and also an internationally renowned academic leader.

[00:00:36] I'm excited to have him here today with us to talk to us about what Wolters Kluwer is up to in the world of AI as well as the business that they're in. Dr. Bonis, thanks for being with us. Thanks for having me, Saul. It's a pleasure. And to kick things off here, why don't you tell us a little bit about your role there and how the company is using digital health technology to drive meaningful change in healthcare?

[00:00:59] You've already given me a generous introduction, so I won't expand on that too much other than just say that I still see patients. I'm on the faculty at House Medical Center. I'm a gastroenterologist, so I'm still out there in the frontline making sure that our organization stays focused on delivering effective solutions for frontline providers and the patients that they serve. So my role, as you mentioned, is as chief medical officer for the health division where I have a role in operations and also on the strategy of the company.

[00:01:26] That's fantastic. That's so great that you still practice too. It keeps you fresh and up to date with the real opportunities and challenges. And so with that end, what would you say is the main way that the company is using digital health and technology to make change in healthcare?

[00:01:42] This was the main reason why I left my position way back when as heading towards a full-time academic practicing and research position because I recognized early on that up to date, which was my entrance into Walters Chloro, was trying to use digital technologies to produce on scale the ability to improve care.

[00:02:02] That was exciting to me. And our ability to do that really on even much larger scale, on a global scale under Walters Chloro's umbrella, has been quite profound and had a measurable impact. So we are focusing on using technology to improve the consistency and quality of care. And we fully recognize that affords opportunities for us to address the major challenges in health services delivery, not just the U.S., but globally, such as affordability, access, equity, and a sustainable financial model.

[00:02:30] So a big part of that and what really excited me when I first joined UpToDate was making high-quality information that's easier to access at or near the point of care and the subsequent actions as much automation as possible.

[00:02:45] So really just to make it easier for healthcare professionals and all of them, so doctors, nurses, advanced practice providers, pharmacists, physicians, assistants, just make it easy for them to do the right thing so that the patients that they're seeing, wherever they're seeing, and whoever they're seen by, have the best possible chance of a favorable outcome. So we have a lot of initiatives in this direction. One of them is UpToDate Expert AI. So we're using advanced technology. We have a long history of using advanced technology.

[00:03:14] So the AI modalities and Gen AI is new, but we've been using this to our advantage for many years. And we've combined that with extensive human oversight, trying to deliver evidence-based clinical content that's efficient and reliable, especially for this really high-stakes domains where we operate. Asal, if you let me just make that point and belabor it just a little bit more, I want the audience to really do the thought experiment and why this is so important.

[00:03:41] So what we know is that healthcare professionals have a lot of questions. They don't always pursue them, but if they do pursue them, about 30% of the time, they'll actually change what they do. So meaning that they have an idea in mind, and if they don't have the right information at hand, they might just do what they always have done or what their colleagues have done. But if you give them the right information in a graceful way, they'll change what they do often.

[00:04:06] And you can imagine that is a tremendous responsibility for those who are providing that information because it directly is going to influence the way patients are cared for. So if you're sitting with your doctor or with a loved one sitting with their doctor, how do you know that they're right? You sure as heck hope that if they're going to look something up, that they're looking it up in a place that's reliable and making the best possible decision since they might be making a different decision based on that information.

[00:04:33] So trust, transparency, proven clinical rigor, that's absolutely fundamental. And that's why I'm really excited because we've grounded our AI efforts in UpToDate, which we have been building for decades against that true north. Well, I really appreciate that. Yeah. When you're at the point of care, the right decisions are oftentimes lifesaving. And so in your mind, what's the most important innovation your team is focused on right now?

[00:05:02] And what problem is it designed to solve? So we are spending a lot of effort focusing on building the most reliable AI experience that we can create for healthcare professionals. And so that's our expert AI, which we have been launching. We have many other initiatives underway so that they're all focusing on workflow of healthcare professionals within the electronic medical records. And we've announced multiple partnerships also with other vendors, like-minded vendors that are within workflow.

[00:05:31] So the idea is, again, to make it very easy using advanced technology to deliver information and the ability to follow up on that information where people are working. The goal here is to recognize that healthcare professionals are adopting these Gen AI tools faster than they can be proven to be safe and effective and very critically fit for duty for the enterprise. And there's reasons to be concerned about that. And we're aware of them and we're trying to avert them.

[00:05:56] So there's accumulating data showing that these systems don't necessarily perform as well as we would hope. So we've been incredibly deliberate about it. We've taken our time. We haven't delivered as fast as other vendors, but that's been on purpose because we've earned trust and we want to make sure what we deliver is worthy of the trust that healthcare professionals and patients have placed in us for decades. That's the biggest focus for us in 26.

[00:06:21] So building a purpose-built alternative, it needs to be fast, transparent, grounded in validated, trustworthy information, and giving clinicians the speed that they want, the efficiency that they want, but without sacrificing reliability, privacy, or trust, or potentially some of the risks to the enterprise itself. Yeah, for sure. It's definitely something that is critical. The safety, the security of how these technologies are used. And you had discussed the importance of governance.

[00:06:52] So talk to us about the topic of shadow AI and what enterprise-level risk does it pose within healthcare environment? Yes. So a neologism, but we didn't coin the phrase. It's been out there now, but shadow AI refers to staff, particularly within enterprises, clinicians, and others who are using AI tools that have not been officially vetted, approved, or secured by the organization. It is widespread.

[00:07:17] We conducted a survey last year, so more than 500 healthcare professionals and administrative staff within enterprises in the United States, and more than half reported encountering unauthorized use. And nearly one in five healthcare professionals admitted that they were using them. That's probably an underestimate. In other studies, it's been much, much higher than that. These are good intentions. People are finding tools that help them do their jobs.

[00:07:40] But what they may not be aware of is that they're introducing risks not only to themselves and to their patients, but also to the enterprise. So for themselves and their patients, it's a matter of the clinical validity and the reliability of the systems and their ability to find faulty information. But for the enterprise is whether or not these systems are compliant with the evolving regulations and also security.

[00:08:03] So on that point, IBM published a fascinating report that we're already seeing shadow AI-related breaches. And those tend to be more costly than previous breaches as well. So these are real risks that the enterprise can experience by not having a clear sight line into what their staff and their providers are using. There's other issues too, like who owns the risk? So that is legal liability. The medical tort system has yet to sort all of that out if something goes wrong.

[00:08:33] But I want to just put this into perspective. It's not all bad. I think the reason that people are looking at these tools and adopting them is because it helps them do their job. So it's shining a light on the unmet needs of individual practitioners and other staff within an enterprise and perhaps then the unmet needs of the enterprise. But really, without governance, it's exposing patients, the clinicians, and the entire enterprise to avoidable risk.

[00:09:00] It's a significant concern because the data involved that we deal with in healthcare is highly regulated. And the consequences of a breach could be really bad. I'm glad we're having this conversation. It's something that we all need to be thinking about. When deploying Gen.AI for high-risk clinical applications like CDS or clinical decision support, what specific risks should leaders anticipate and address? So there's a bunch of them.

[00:09:29] And if we look at the entire healthcare ecosystem, but even more broadly, the AI ecosystem, the governance is really just getting going. It's not well established yet. And of course, enterprises that have more resources are better able to do governance themselves. Others might be relying on the vendors themselves or potentially their electronic medical record system itself to vet the AI solutions.

[00:09:53] But by and large, leaders need to look beyond all the hype and to focus squarely on safety, reliability, oversight, and the value proposition to the enterprise. So for key risks in areas where we operate, which is very high stakes, frontline provider decisions, include incorrect or outdated recommendations.

[00:10:15] What we found in our data, we hope to publish this, is that when we expose even experts, so specialists, to faulty information created by Gen.AI relative to a reference standard, about half of them actually didn't recognize that the information was faulty. So the degree to which even experts can intercept faulty information is really uncertain. We know from many published studies now that the outputs can be biased, which may disproportionately affect vulnerable populations.

[00:10:43] There may be the risk of over-reliance, this automaticity where people do what the AI says without double-checking it or really engaging in a thought process. We've seen some examples of this where drugs that were unsafe, for example, during pregnancy were recommended because the AI nor the provider queried about pregnancy status. And then there's also the concern, not yet extensively studied, but a real concern around clinical de-skilling, particularly among trainees, but everybody who's using these tools.

[00:11:12] So many of the AI tools are focusing on addressing these concerns with explainability, but that too is variable. So if clinicians can't really understand or trace the recommendations where they came from, and realistically, even though many say that they might go down and look at the primary references, but even better, the body of literature on that, can we realistically expect them to do that when they're busy and time-pressed? But all of that has the potential to weaken trust and compromise that decision-making.

[00:11:41] So back into that 30% of times they'll change their mind, what happens when these things go sideways for it? So what we're trying to do is to anchor our AI experience in up-to-date, which we built over many years, 7,600 expert clinicians who contribute to it, an extensive editorial and updating process. We're not trying to absorb the chaos of the intranet to make our recommendations.

[00:12:08] So as leaders try to vet these solutions, they have to look at them from multiple angles. Are they fit for purpose? Can they prove that? Do they really deliver value? What about compliance? And what about security? And how do you actually do that dynamically? Because these things, the technology is evolving and the needs are changing, and the workflows are also evolving. So this cannot be a one and done. It definitely can't. It definitely can't. And I really appreciate your perspective on this one.

[00:12:36] At the end of the day, governance is the key. And so how should leaders manage AI governance to better support clinical decision-making across their organization? So I think the systems as a whole, as I mentioned, is heterogeneous now in the ability to really look at just the myriad of new AI solutions which are being proposed and, again, which are being adopted spontaneously by the staff. But yet that is the task at hand. And so the first is, of course, to establish what the goals are.

[00:13:06] And health systems are trying to deliver safe, consistent, high-quality care and to have the ability to do that in a sustained way. So sustainable financial models. So understanding the objectives. So this is not a technology-led set of objectives. Rather, it is a set of objectives which can be enabled with technology. And then they need to pair that with clear policies and provide the tools that clinicians want to use. So they need to set the standards.

[00:13:35] They need to understand transparency and privacy for the acceptable use of these tools to really help. Frontline clinicians aren't going to be able to vet matters of security and whether or not these solutions are really ready for the enterprise. So enterprises really have to be active in doing that on behalf of their clinicians. And they also need to make sure that they're communicating that.

[00:13:58] In one of the surveys that we did, we found that only 18% of providers associated with enterprise even knew of any AI policies related to that. And finally, the point that I already made is frontline providers are the ones who are at that interface of care. And that governance policy, whatever it is, has to be iterative.

[00:14:19] That frontline needs to get involved and needs to get involved in an ongoing basis as to help make decisions and to vet the solutions that are out there and to make changes when those are necessary. Thank you for that. Yeah, no, it's something that is important for all of us to know. And it sounds like you guys are creating a lot of education around the topic, Peter. So really appreciate you sharing these insights and the work that Wolters Kluwer is doing and also within the up-to-date platform.

[00:14:49] If people want to learn more about you as well as the work that Wolters Kluwer is up to, where can they reach out? I'm available personally on LinkedIn. And as far as up-to-date is concerned, we are part of the Wolters Kluwer ecosystem and we have a fairly extensive presence on the web and on social media. So we should be pretty easy to find. Outstanding. Well, there you have it, folks. An incredible conversation with Dr. Peter Bonis. He's a chief medical officer at Wolters Kluwer.

[00:15:17] In the show notes, you'll find all the short notes on the insights that we discussed with Dr. Bonas today, as well as all the ways to get in touch. Please make sure you share this episode with your friends because this topic on shadow AI is one that needs to be talked about and that we need to be approaching very carefully. I appreciate the thought leadership that Dr. Bonas came with us today to share with all of you. Thank you all for tuning in. And Peter, thank you so much for spending time with us.

[00:15:47] Thanks so much, Saul.