Identifying Emergent AI Modalities to Optimize Oncology Practice

Arturo Loaiza-Bonilla, MD, MSEd, FACP, systemwide chief of Hematology and Oncology at Saint Luke’s University Health Network

Arturo Loaiza-Bonilla, MD, MSEd, FACP, systemwide chief of Hematology and Oncology at Saint Luke’s University Health Network, spoke with CancerNetwork® about emerging modalities for artificial intelligence (AI)-powered technologies that he believes have the potential to transform clinical practice in oncology.

Loaiza-Bonilla began by describing different notable AI pathology and multimodal approaches. Regarding pathology, he explained that AI-powered computer vision enables single-cell sequencing, which can detect minute details within cell microenvironments that the human eye alone cannot perceive. Additionally, he expressed that combined pathology and genomics data create foundational models that help to cluster patient data. Loaiza-Bonilla additionally suggested that the technology should be representative of all subtypes of patients, highlighting a call to action to build massive foundational models assessing billions of features.

To underscore the importance of AI-assisted clinical documentation, Loaiza-Bonilla stated that documentation was a pain point for clinicians, who are caught up with processes that take time away from seeing patients. He then touched upon data from 2 studies, one of which suggests many early adopters of clinical documentation did not see time savings, and the other suggesting that AI documentation improved accuracy and reduced documentation time.

Regarding large language models (LLMs), Loaiza-Bonilla highlighted data indicating that these models could pass every board exam at the highest levels, as well as attain historically high accuracy on Humanity’s Last Exam, a benchmark used for evaluating LLM capabilities. In addition, he expressed that these models were enabling the translation of jargon to accessible language, which may help with patient education and expedite processes that correlate with better patient outcomes. He concluded by suggesting that LLMs will help enhance processes that humans are not naturally proficient with, but that humans will continue to adopt roles that they have naturally evolved proficiency for.

CencerNetwork: What are different digital pathology and multimodal approaches that have the potential to transform clinical practice?

Loaiza-Bonilla: Diagnostics and pattern recognition are things that AI is pretty [effective with]...We knew that AI was going to impact us all when the models were able to solve this challenge called the ImageNet challenge when it came up. AlexNet [then] came up and was able to [perform] better than humans. Now, we have the ability to post a picture and [have] any of these tools recognize and describe it in full detail. Pathology, being something that looks at patterns within our brains, can really help us. In using computer vision, doing segmentation of what we call kernels, which are like the pixels that comprise images on pathology when we do whole-slide uploading in these models, is very promising.

We can go from a spatial transcriptomics to single-cell sequencing; we are now seeing a major impact on these. For example, [we were] curious about how patients may benefit from immunotherapy because we have changes in the microenvironment based on subtypes of lymphocytes. Now, we are using data pathology to give us those biomarkers that sometimes were unknown to us; they are giving us new insights on things that we were not aware of. We are making an impact on response to treatment. Our eyes are amazing tools that have evolved for all of human history, but we know now that when you put billions of data points in terms of visual pixels, we can start seeing things that even our eyes and brains typically do not process. They may process them, but they do not surface because we do not know they exist.

Things that we never even thought were possible before can now be found with these multimodal approaches in pathology. The idea is to combine pathology images with genomics and clinical data so that we can develop these multimodal predictive and prognostic approaches––called foundational models––which are having tons of patients put under this analysis and being able to start putting those clusters of patients and data together.

There are several key emerging platforms. Across more than 17 tumor tissues and many slides—millions of them—we can build these foundational models that are going to, in the future, be standalone and part of the baseline of how we make predictions for patients with cancer. The key thing, now, is making sure that we have an accurate representation of all kinds of patients because the data could be skewed depending on the population of patients. We all have different microenvironments, [as well as] different germline and somatic genomics. This is a call to action to build huge foundational models in the billions the same way as LLMs, where billions of tokens, features, or parameters [were assessed]. We can do the same in the billions of giga pixels or clusters.

Are there any relevant data that speak to the benefit of AI use in clinical documentation?

Loaiza-Bonilla: Clinical documentation is a major pain point for us clinicians. We get tired of doing an extra click or putting in an extra note for something that we know is self-evident, but if it does not get put in the system, we do not get billed, and that is the nature of what we practice. Now, we are trying to rely on data summarization, the same process that I use for clinical trials. We can do it as well for usual standard of care, but we are now also relying on new things called ambient AI scribes.

There are different companies doing this [such as] Microsoft and Abridge. The idea is that they want to reduce documentation time while we talk to patients. The studies show that these tools may decrease burnout to a certain extent––to how much is hard to quantify because everyone has different levels of burnout.

There are a couple of studies. One trial [published in JAMA] revealed that some early adopters saw no time savings, approximately 44% of them.¹ But that may [suggest] that we need to do some fine-tuning there, that we need to optimize the time invested in more administrative tasks. That time is off; we should be focused on taking care of patients.

In technicality, an analysis of 36 studies in 2024 found that AI [improved documentation] accuracy between 22% and 29% and was able to decrease the time spent [documenting].² Now, the focus is fine-tuning it for the specialty use case in surgery, radiation oncology, radiology, and medical oncology. [We had the start we needed], and we have a good foundation at this point.

How might LLMs be employed to enhance clinical practice for patients?

Loaiza-Bonilla: As mentioned, LLMs and language models have made a lot of difference in clinical practice and even medical education. Now, we know that these LLMs can pass every board exam that we have at a high level, even without specialization. Now, we have the OpenAI o3 model that was just released. There is this test called Humanity’s Last Exam. In the past, no LLM was able to go over approximately 10%; this one went to [13%, and OpenAI’s new Deep Research agentic AI tool reached] 26.6%.³ We are getting close to it, and this is a complex exam.

Of course, medicine is complex, but we are task-oriented. Not everyone does everything. I do not do neurosurgery; we need training for that, and we need different agents for that. Now, we are focusing on how we can use LLMs to optimize, for example, patient education. How can we use it for translating complex jargon into accessible language for our patients so they understand what we are telling them, [leading to] more adherence and more trust [while] leveraging the empathetic component that was taught by us?

[A patient might] say, “oh, LLMs are nicer, and they are more empathetic than physicians.” Well, if you give me time, I can be as empathetic because we were the ones who trained the systems to be like that. It is not that it came out of nowhere. As far as I know, it is all coming from us humans. The idea here is that we want to use this to synthesize information, to [provide] summarization, give us more insights on things that are missing, improve clinical decision support, and [perform] simulated cases where we can say, “Okay, you have seen this case before,” like a tumor board to see how those patients did based on where they fit.

Clinical practice can be optimized in multiple ways. The difficult part is more about the quantification of it. If the LLMs and all these models and algorithms are helping the patient get access to the physician on the phone––we have an agent that answers the phone and is able to talk and get you to the [correct] operator––it helps to make your infusion time better because they are able to look at those numbers. They are able to order the test on behalf of the physician. They click a button, and then you talk to the patient for more time because you have the time to do so by documentation. It also helps radiologists, pathologists, and radiation oncologists do better contouring on the radiation suite, it helps the surgeon be more accurate in terms of margins, [and it helps] the pathologist detect those margins or biomarkers faster and receive the right drug in the clinical trials. All that is compounded interest.

The impact is going to be big, but we need an AI to quantify it for us. It is so much complexity, and we live in our little silos that [make it complicated] to see the use cases as a whole. We need that NASA command center approach to look at the implementation of AI in clinical practice; I feel that there are many things we can leverage for the positive end.

One point I wanted to make is, recently, there was this New York Times editorial by Eric Topol, MD, and Pranav Rajpurkar, PhD. The idea was, “Is the robot ready to see you in clinic?”⁴ The analysis [showed] that AI is sometimes better than us, and we have to accept that fact. We have to relinquish some things so we can focus on other things that are harder for machines. That is called Moravec’s Paradox. That is a concept from the 1980s where we have evolved certain things we are much better in vs AI in terms of finesse, touching, and performing procedures [rather] than doing mathematics, which is a much more recent concept for humans. We did not have the evolutionary pressure to be good at math. LLMs can do that much faster and further. We want Moravec’s Paradox to be resolved with us, not being under the belief that we are better than the machines in certain tasks.

References

1. Liu T-L, Hetherington TC, Stephens C, et al. AI-powered clinical documentation and clinicians' electronic health record experience: a nonrandomized clinical trial. JAMA Netw Open. 2024;7(9):e2432460. doi:10.1001/jamanetworkopen.2024.32460
2. Lee C, Britto S, Diwan K. Evaluating the impact of artificial intelligence (AI) on clinical documentation efficiency and accuracy across clinical settings: a scoping review. Cureus. 2024;16(11):e73994. doi:10.7759/cureus.73994
3. Okemwa K. OpenAI's new "Deep Research" blows ChatGPT o3-mini and DeepSeek out of the water with 26.6% accuracy in the world's hardest "AI exam" — but it skipped the line. News release. Windows Central. February 5, 2025. Accessed February 27, 2025. https://tinyurl.com/3nt59tmh
4. Topol E, Rajpurkar P. The robot doctor will see you now. New York Times. February 2, 2025. Accessed February 27, 2025. https://tinyurl.com/kaeu7p7d