REALM: RAG-Driven Enhancement of Multimodal Electronic Health Records Analysis via Large Language Models
Institute of Artificial Intelligence, Beihang University(北京航空航天大学人工智能研究院);School of Computer Science and Engineering, Beihang University(北京航空航天大学计算机科学与工程学院);School of Software, Beihang University(北京航空航天大学软件学院);China Mobile Research Institute(中国移动研究院)
Yinghao Zhu, Changyu Ren, Shiyun Xie, Shukai Liu, Hangyuan Ji, Zixiang Wang, Tao Sun, Long He, Zhoujun Li, Xi Zhu, Chengwei Pan
The integration of multimodal Electronic Health Records (EHR) data has significantly improved clinical predictive capabilities. Leveraging clinical notes and multivariate time-series EHR, existing models often lack the medical context relevent to clinical tasks, prompting the incorporation of external knowledge, particularly from the knowledge graph (KG). Previous approaches with KG knowledge have primarily focused on structured knowledge extraction, neglecting unstructured data modalities and semantic high dimensional medical knowledge. In response, we propose REALM, a Retrieval-Augmented Generation (RAG) driven framework to enhance multimodal EHR representations that address these limitations. Firstly, we apply Large Language Model (LLM) to encode long context clinical notes and GRU model to encode time-series EHR data. Secondly, we prompt LLM to extract task-relevant medical entities and match entities in professionally labeled external knowledge graph (PrimeKG) with corresponding medical knowledge. By matching and aligning with clinical standards, our framework eliminates hallucinations and ensures consistency. Lastly, we propose an adaptive multimodal fusion network to integrate extracted knowledge with multimodal EHR data. Our extensive experiments on MIMIC-III mortality and readmission tasks showcase the superior performance of our REALM framework over baselines, emphasizing the effectiveness of each module. REALM framework contributes to refining the use of multimodal EHR data in healthcare and bridging the gap with nuanced medical context essential for informed clinical predictions.
Rethinking All Evidence: Enhancing Trustworthy Retrieval-Augmented Generation via Conflict-Driven Summarization
University of Chinese Academy of Sciences(中国科学院大学);Chinese Academy of Sciences(中国科学院);National University of Defense Technology(国防科技大学);Chongqing University of Technology(重庆理工大学)
RAG-based EEG-to-Text Translation Using Deep Learning and LLMs
IAS-LAB, Department of Information Engineering, University of Padova(帕多瓦大学信息工程系IAS实验室);Padova Neuroscience Center(帕多瓦神经科学中心);Department of Health Technology, Technical University of Denmark(丹麦技术大学健康技术系)
CONFLARE: CONFormal LArge language model REtrieval
Mayo Clinic Artificial Intelligence Laboratory, Mayo Clinic(梅奥诊所人工智能实验室,梅奥诊所);Orthopedic Surgery Artificial Intelligence Laboratory, Mayo Clinic(梅奥诊所骨科手术人工智能实验室);Tehran University of Medical Sciences(德黑兰医科大学)
RGAR: Recurrence Generation-augmented Retrieval for Factual-aware Medical Question Answering
King’s College London(伦敦国王学院);School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University(上海交通大学电子信息与电气工程学院);School of Electrical and Computer Engineering, Carnegie Mellon University(卡内基梅隆大学电气与计算机工程学院)
Sentence-Level Context Sensitivity as a Training-Free Detector of Unsupported Content, Evaluated Against Trained Verifiers
Centre for Intelligent Cloud Computing, CoE for Advanced Cloud, Faculty of Information Science and Technology, Multimedia University(智能云计算中心、高级云研究中心、信息科学与技术学院、多媒体大学)
Evaluating Retrieval-Augmented Generation vs. Long-Context Input for Clinical Reasoning over EHRs
University of Wisconsin-Madison(威斯康星大学麦迪逊分校);Loyola University Chicago(芝加哥洛约拉大学);Boston Children’s Hospital Harvard Medical School(波士顿儿童医院哈佛医学院);University of Colorado-Anschutz(科罗拉多大学安舒茨分校)
COMPARE: Clinical Optimization with Modular Planning and Assessment via RAG-Enhanced AI-OCT: Superior Decision Support for Percutaneous Coronary Intervention Compared to ChatGPT-5 and Junior Operators
Tangdu Hospital, The Fourth Military Medical University(空军军医大学唐都医院);Research Institute of Tsinghua University in Shenzhen(深圳清华大学研究院);The Second Affiliated Hospital, Air Force Medical University(空军军医大学第二附属医院)
RDMA: Cost Effective Agent-Driven Rare Disease Mining from Electronic Health Records
Department of Computer Science, University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校计算机科学系);Department of Pediatrics, University of Illinois College of Medicine Peoria(伊利诺伊大学皮奥里亚医学院儿科系)
ColaCare: Enhancing Electronic Health Record Modeling through Large Language Model-Driven Multi-Agent Collaboration
Peking University(北京大学);ETH Zurich(苏黎世联邦理工学院);Peking University Third Hospital(北京大学第三医院);University of Edinburgh(爱丁堡大学);Beihang University(北京航空航天大学);Health Data Research UK(英国健康数据研究中心)
Tool-calling retrieval versus vector RAG for a small Greek--English knowledge base: accuracy and robustness to how users type Greek
Institute of Communication and Computer Systems (ICCS)(通信与计算机系统研究所);National Technical University of Athens(雅典国立技术大学);University of Thessaly(色萨利大学);DASKALOS-APPS(DASKALOS-APPS公司)
VISTA Architect: A graph database-oriented health AI system demonstrated in multidisciplinary tumor boards
Department of Biomedical Data Science, Stanford University School of Medicine(斯坦福大学医学院生物医学数据科学系);Department of Medicine, Stanford University School of Medicine(斯坦福大学医学院医学系)