Predicting Cardiovascular Readmission Risk from Doctor-Patient Conversation

Motivation

Finding complex patterns in data for predicting clinical outcomes is highly challenging, especially as the volume of clinical data increases exponentially in the era of big data.

Large language models (LLMs) are general-purpose models for analyzing large quantities of textual data. Further, LLMs can be fine-tuned (i.e., tailored) to serve the specific needs of a wide variety of tasks, including in healthcare.

LLMs can be fine-tuned to operate at varying levels of textual granularity (e.g., individual words, full sentences, or entire conversations). In addition, LLMs can be fine-tuned to: recognize key named entities, such as symptoms, diseases, medications, treatments, procedures, or timelines; extract events and relations—such as cause, precedence, or synonymy—between these key entities; and analyze the sentiment (i.e., tone/mood) of each sentence, speaker, or conversation. LLMs can also integrate external expert knowledge, such as biomedical terminology databases, in their reasoning.

Overview

We seek to predict outcomes of clinical interest, such as 30-day readmission risk, from transcripts of recorded doctor-patient interviews and associated structured electronic medical record (EMR) data using state-of-the-art LLMs.

The key goals of the project are to:

• Build AI-based prediction models for each outcome of interest.
• Validate the accuracy and effectiveness of the built models, and compare them against more traditional methods.
• Present a framework for the implementation and deployment of the models in a clinical setting as a means of providing clinical decision support.

Several BERT variants, such as BioBERT, ClinicalBERT, and PubMedBERT, are specifically geared towards the analysis of medical text. These serve as our baseline models. State-of-the-art LLMs, such as Llama-3 70B, are significantly more powerful, but also less tuned to the medical domain. Tailoring state-of-the-art LLMs to our use case is a critical step in this project.

In addition, we are extracting clinically-relevant features using modular biomedical NLP techniques—named entity recognition, event and relation extraction, sentiment analysis, knowledge integration, and others. These serve as inputs to more traditional ML/DL-based classifiers, as additional comparators and baselines.

Experimental Results

This project is ongoing! Stay tuned for results…