Multi-modal Clinical Diagnostic Reasoning

Diagnostic errors affect 12 million patients in the U.S. and contribute to 80,000 deaths per year. The main causes for diagnostic errors include cognitive biases introduced by healthcare providers, miscommunication between healthcare teams, lack of access to key data, and not recognizing time-sensitive data in the electronic health record (EHR). The cognitive burden from information overload in the EHR cause clinicians to take decisional shortcuts with biased heuristics and miss critical data in the EHR, leading to missed opportunities for timely and accurate diagnoses. Artificial Intelligence (AI) and clinical Natural Language Processing (cNLP) provide opportunity to help understand medical text and can automate EHR analysis, pointing to the promising direction of invoking medical knowledge and clinical experience as humans do. However, the majority of the cNLP tasks are not designed for bedside application to generate diagnoses and augment bedside decision-making.

We have have gathered preliminary data and designed cNLP benchmark tasks for clinical diagnostic reasoning. Our tasks address key cognitive processes to build models in this proposal that can synthesize EHR data to generate diagnoses that align with evidence-based medicine and medical knowledge representation. Read about the first diagnostic reasoning NLP benchmark.

In the next stage, we will explore the capabilities of large language models (LLMs) in predicting diagnoses using both structured and unstructured text. Our approach involves building a neural symbolic knowledge base that will enhance the LLM’s diagnostic process within a Retrieval-Augment-Generation framework. This framework aims to achieve knowledge grounding and reduce hallucination in the models’ outputs, ensuring more reliable and accurate predictions. Concurrently, we aim to conduct a physician-centered evaluation to assess the efficacy of LLMs in providing diagnostic decision support. This evaluation will focus on how well the LLMs integrate and interpret clinical data, potentially transforming the landscape of medical diagnostics.

This research is a K99/R00 project funded by National Instiute of Health, National Library of Medicine (1K99LM014308-01).