Large Language Models (LLMs) have demonstrated impressive capabilities in role-playing scenarios, particularly in simulating domain-specific experts using tailored prompts. This ability enables LLMs to adopt the persona of individuals with specific backgrounds, offering a cost-effective and efficient alternative to traditional, resource-intensive user studies. By mimicking human behavior, LLMs can anticipate responses based on concrete demographic or professional profiles. In this paper, we evaluate the effectiveness of LLMs in simulating individuals with diverse backgrounds and analyze the consistency of these simulated behaviors compared to real-world outcomes. In particular, we explore the potential of LLMs to interpret and respond to discharge summaries provided to patients leaving the Intensive Care Unit (ICU). We evaluate and compare with human responses the comprehensibility of discharge summaries among individuals with varying educational backgrounds, using this analysis to assess the strengths and limitations of LLM-driven simulations. Notably, when LLMs are primed with educational background information, they deliver accurate and actionable medical guidance 88% of the time. However, when other information is provided, performance significantly drops, falling below random chance levels. This preliminary study shows the potential benefits and pitfalls of automatically generating patient-specific health information from diverse populations. While LLMs show promise in simulating health personas, our results highlight critical gaps that must be addressed before they can be reliably used in clinical settings. Our findings suggest that a straightforward query-response model could outperform a more tailored approach in delivering health information. This is a crucial first step in understanding how LLMs can be optimized for personalized health communication while maintaining accuracy.
MedDiT: A Knowledge-Controlled Diffusion Transformer Framework for Dynamic Medical Image Generation in Virtual Simulated Patient
Wangxuan Institute of Computer Technology, Peking University(北京大学王选计算机研究所);Pattern Recognition Center, WeChat AI, Tencent Inc(腾讯公司微信人工智能模式识别中心);School of Computer Science, Wuhan University(武汉大学计算机学院)
Leveraging Large Language Model as Simulated Patients for Clinical Education
Wangxuan Institute of Computer Technology, Peking University(北京大学王选计算机研究所);School of Computer Science, Wuhan University(武汉大学计算机学院);CureFun Co.(CureFun公司)
LLMs Can Simulate Standardized Patients via Agent Coevolution
State Key Lab of CAD & CG, Zhejiang University(浙江大学计算机辅助设计与图形学国家重点实验室);Zhejiang Polytechnic Institute, Polytechnic Institute, Zhejiang University(浙江工业大学之江学院);College of Computer Science & Technology, Zhejiang University(浙江大学计算机科学与技术学院);East China Normal University(华东师范大学);Sun Yat-sen University Cancer Center(中山大学肿瘤中心);School of Public Health, Zhejiang University(浙江大学公共卫生学院);Second Affiliated Hospital, Zhejiang University School of Medicine(浙江大学医学院第二附属医院)
PaniniQA: Enhancing Patient Education Through Interactive Question Answering
University of Massachusetts Amherst(马萨诸塞大学阿默斯特分校);Emory University(埃默里大学);Northeastern University(东北大学);UMass Chan Medical School(马萨诸塞大学陈医学院);University of Minnesota(明尼苏达大学);University of Massachusetts Lowell(马萨诸塞大学洛厄尔分校)
Simulated patient systems powered by large language model-based AI agents offer potential for transforming medical education
The Chinese University of Hong Kong(香港中文大学);University of Michigan(密歇根大学);Stanford University(斯坦福大学);Mass General Brigham(麻省总医院布莱根医疗体系);Harvard Medical School(哈佛医学院);Cornell University(康奈尔大学);Peking University Third Hospital(北京大学第三医院);Shandong University(山东大学);Qilu Hospital of Shandong University(山东大学齐鲁医院);Rutgers University(罗格斯大学);Broad Institute of MIT and Harvard(麻省理工学院与哈佛大学布罗德研究所);The University of Hong Kong(香港大学);Chinese Academy of Medical Sciences and Peking Union Medical College(中国医学科学院北京协和医学院);Chinese Academy of Medical Sciences(中国医学科学院);University of Ottawa(渥太华大学);Peking University Sixth Hospital(北京大学第六医院)
Comparisons between a Large Language Model-based Real-Time Compound Diagnostic Medical AI Interface and Physicians for Common Internal Medicine Cases using Simulated Patients
Shihwa Medical Center(始华医疗中心);Helpmedoc Inc.(Helpmedoc公司);Asan Medical Center(峨山医疗中心);University of Ulsan College of Medicine(蔚山大学医学院)
本研究开发了基于大语言模型的实时复合诊断医疗AI界面,通过非随机临床试验与医生比较,发现其在常见内科病例中诊断准确率更高(80% vs 50-70%)、时间更短(缩短44.6%)、成本更低(降低98.1%),患者满意度相当,显示出辅助初级保健的潜力。
The complexities of patient-centred conversational artificial intelligence
Verily Health(Verily健康公司);University of Oxford(牛津大学);University of Birmingham(伯明翰大学);University Hospitals Birmingham NHS Foundation Trust(伯明翰大学医院NHS基金会信托);Harvard Medical School(哈佛医学院);Beth Israel Deaconess Medical Center(贝斯以色列女执事医疗中心)
GlyTwin: Digital Twin for Glucose Control in Type 1 Diabetes Through Optimal Behavioral Modifications Using Patient-Centric Counterfactuals
College of Health Solutions, Arizona State University(亚利桑那州立大学健康解决方案学院);School of Computing and Augmented Intelligence, Arizona State University(亚利桑那州立大学计算与增强智能学院);Department of Endocrinology, Mayo Clinic Arizona(梅奥诊所亚利桑那分部内分泌科)
Directive Explanations for Monitoring the Risk of Diabetes Onset: Introducing Directive Data-Centric Explanations and Combinations to Support What-If Explorations
Predicting Therapeutic Outcome via Aligning Patient-Specific Knowledge Graph and Gene-Level Perturbation Representations
Interdisciplinary Program in Bioinformatics, Seoul National University(首尔国立大学生物信息学跨学科项目);AIGENDRUG Co., Ltd.(爱真药物有限公司);BK21 FOUR Intelligence Computing, Seoul National University(首尔国立大学BK21四号智能计算);Interdisciplinary Program in Artificial Intelligence, Seoul National University(首尔国立大学人工智能跨学科项目);Department of Artificial Intelligence, Inha University(仁荷大学人工智能系)