SIR 2024
Interventional Oncology
Alexander Shieh, MD
Postdoctoral Fellow
MD Anderson Cancer Center
Financial relationships: Full list of relationships is listed on the CME information page.
Iwan Paolucci, Ph.D.
Postdoctoral Fellow
The University of Texas MD Anderson Cancer Center
Disclosure information not submitted.
Jessica Albuquerque Marques Silva, MD
Postdoctoral Fellow
Department of Interventional Radiology, MD Anderson Cancer Center
Disclosure information not submitted.
Bruno C. Odisio, MD, FSIR (he/him/his)
Interventional Radiologist
MD Anderson Cancer Center
Disclosure information not submitted.
Procedural registries are essential for accumulating clinical evidence and quality improvement in interventional oncology. Currently, registries are built upon manual review of post-interventions follow-up images, a time-consuming specialized task that limits widespread adoption of such registries. Here, we used a large language model (LLM) to read free-text diagnostic imaging reports of patients who underwent percutaneous ablation (PTA) of hepatocellular carcinoma (HCC) and automatically extract critical findings that suggest cancer progression.
Materials and Methods: A total of 47 consecutive patients receiving PTA for HCC during 2022-2023 were obtained from our single-center, HIPPA compliant, IRB-approved retrospective PTA registry with a waiver of informed consent. Post-PTA CT and MRI reports for HCC evaluation were collected. The open-source Llama 2 LLM{1} was installed in internal servers to ensure the safety and compliance with IRB and HIPPA regulations. We designed a prompt to extract critical cancer progression findings without further training and calculated its precision and recall. A sub-analysis on cancer progression patterns was performed on the following: local disease progression (LDP, local tumor progression or residual unablated tumor), intrahepatic progression (IHP, progression within liver excluding LDPs) and extrahepatic progression (EHP). Ground truths were obtained by manual annotation. For reports using Liver Imaging Reporting and Data System (LI-RADS){2}, we defined LR and TR viable lesions as LDP, and LR4 and LR5 lesions as IHP. Progression finding is defined as new or enlarging lesions compared to pre-PTA scan reports.
Results: A total of 133 reports were analyzed (97 [73%], CT; 36 [27%], MRI). LI-RADS was utilized in 39 (29%) reports. Manual annotation disclosed 148 cancer progression findings . The overall recall for LLM extraction was 0.79, and the overall precision was 0.53. Sub-analysis of recall and precision were 0.31 and 0.19 for LDP, 0.76 and 0.64 for IHP, and 1.0 and 0.34 for EHP. The recall was similar for reports with and without LI-RADS (LDP: 0.33 vs 0.29; IHP: 0.72 vs 0.78). However, the precision is much higher in reports with LI-RADS findings (LDP: 0.29 vs 0.14; IHP: 0.85 vs 0.58).
Conclusion: We demonstrated the feasibility of using LLM to extract cancer progression findings following PTA of HCCs. This system may facilitate automated registry construction for interventional oncology procedures, flagging sentinel scans for thorough review. Our results also suggested reporting with LI-RADS leads to more precise extraction by LLMs.