Large language models are increasingly being used in patient-facing medical question answering, where hallucinated outputs can vary widely in potential harm. However, existing hallucination standards and evaluation metrics focus primarily on factual correctness, treating all errors as equally severe. This obscures clinically relevant failure modes, particularly when models generate unsupported but actionable medical language. We propose a risk-sensitive evaluation framework that quantifies hallucinations through the presence of risk-bearing language, including treatment directives, contraindications, urgency cues, and mentions of high-risk medications. Rather than assessing clinical correctness, our approach evaluates the potential impact of hallucinated content if acted upon. We further combine risk scoring with a relevance measure to identify high-risk, low-grounding failures. We apply this framework to three instruction-tuned language models using controlled patient-facing prompts designed as safety stress tests. Our results show that models with similar surface-level behavior exhibit substantially different risk profiles and that standard evaluation metrics fail to capture these distinctions. These findings highlight the importance of incorporating risk sensitivity into hallucination evaluation and suggest that evaluation validity is critically dependent on task and prompt design.
ChatDoctor: A Medical Chat Model Fine-Tuned on a Large Language Model Meta-AI (LLaMA) Using Medical Domain Knowledge
University of Texas Southwestern Medical Center(德克萨斯大学西南医学中心);University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);The Ohio State University(俄亥俄州立大学)
Dr. Bias: Social Disparities in AI-Powered Medical Guidance
Mila - Quebec AI Institute(魁北克人工智能研究所);Centre for Advanced Studies in Bioscience Innovation Law, Faculty of Law, University of Copenhagen(哥本哈根大学法学院生物科学创新研究中心);Center for the Study of Democratic Citizenship, Department of Political Science, McGill University(麦吉尔大学政治科学系民主公民研究所以及)
Large language models provide unsafe answers to patient-posed medical questions
Glass Box Medicine;University of Virginia(弗吉尼亚大学);Riverside Community Hospital(河滨社区医院);Inova Schar Heart and Vascular(Inova Schar 心脏与血管研究所);Inova Fairfax Medical Campus(Inova Fairfax 医疗园区);Cooper University Hospital(库珀大学医院);South Texas Research Institute(南德克萨斯研究所);University of California(加利福尼亚大学);Valley View Hospital(Valley View 医院);University of Louisville(路易斯维尔大学);University of Oklahoma Health Sciences Center(俄克拉荷马大学健康科学中心);Northwell Health;Emory University(埃默里大学);Brigham and Women's Hospital(布莱根妇女医院);Castilleja School(卡斯蒂列亚学校);Piedmont Healthcare;AmyMD Wellness
One Size Fits None: Heuristic Collapse in LLM Investment Advice
Computer Science and Artificial Intelligence Laboratory(计算机科学与人工智能实验室);Massachusetts Institute of Technology(麻省理工学院);Sloan School of Management(斯隆管理学院)
Dr.Copilot: A Multi-Agent Prompt Optimized Assistant for Improving Patient-Doctor Communication in Romanian
National University of Science and Technology POLITEHNICA Bucharest(波兰技术科学大学);Dalle Molle Institute for Artificial Intelligence Research (IDSIA)(达勒莫利人工智能研究所);MedicChat
A Survey on Large Language Models from General Purpose to Medical Applications: Datasets, Methodologies, and Evaluations
School of Computer & Communication Engineering, University of Science and Technology Beijing(北京科技大学计算机与通信工程学院);School of Computer Science, University of South China(南华大学计算机学院);Faculty of Computer and Information Sciences, Hosei University(法政大学计算机与信息科学学院)
Clinical knowledge in LLMs does not translate to human interactions
Oxford Internet Institute(牛津互联网研究所);University of Oxford(牛津大学);Nuffield Department of Primary Care(初级保健部门);North Wales Medical School(北威尔士医学院);Bangor University(班戈大学);Contextual AI(上下文AI);MLCommons;Factored AI;Birmingham Women’s and Children’s NHS Foundation Trust(伯明翰妇女和儿童国家卫生信托基金会);Institute of Biomedical Engineering(生物医学工程研究所)
OncoGPT: A Medical Conversational Model Tailored with Oncology Domain Expertise on a Large Language Model Meta-AI (LLaMA)
Shenzhen Kanghua Juntai Biotech Co. Ltd.(深圳康华君泰生物科技有限公司);University of California, Davis(加利福尼亚大学戴维斯分校);The First Affiliated Hospital of Jinan University(暨南大学附属第一医院)
Pattern Recognition or Medical Knowledge? The Problem with Multiple-Choice Questions in Medicine
Université catholique de Louvain(鲁汶天主教大学);Louvain Research Institute in Management and Organizations(鲁汶管理与组织研究所);Cliniques universitaires Saint-Luc(圣吕克大学诊所)
Confidence Calibration for Multimodal LLMs: An Empirical Study through Medical VQA
College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院);School of Computer Science and Technology, Xidian University(西安电子科技大学计算机科学与技术学院);Zhihui Medical Technology (Shanghai) Co., Ltd.(智汇医疗科技(上海)有限公司)
Marking the Wrong Symptoms: Evaluating LLM Watermarks in Medical Texts
ETH Zurich(苏黎世联邦理工学院);Berlin Institute of Health at Charité – Universitätsmedizin Berlin(柏林夏里特医学院柏林健康研究所);Deutsches Herzzentrum der Charité – Medical Heart Center of Charité and German Heart Institute Berlin(柏林夏里特医学院德国心脏中心与柏林德国心脏研究所)
LLMs for Medical Consultation Are Evaluated Too Late: The Preformulation Gap
Harvard T.H. Chan School of Public Health(哈佛陈曾熙公共卫生学院);Harvard Medical School(哈佛医学院);Australian Artificial Intelligence Institute, University of Technology Sydney(悉尼科技大学澳大利亚人工智能研究所);Raycaster;Beth Israel Deaconess Medical Center(贝斯以色列女执事医疗中心)
Topics as Proxies for Sociodemographics: How Conversational Context Affects LLM Answers
Institute for Logic, Language and Computation, University of Amsterdam(逻辑、语言与计算研究所,阿姆斯特丹大学);Khoury College of Computer Sciences, Northeastern University(计算机科学学院,东北大学);Center for Language and Cognition, University of Groningen(语言与认知中心,格罗宁根大学)
A Systematic Analysis of Declining Medical Safety Messaging in Generative AI Models
Department of Medicine, University of British Columbia(英属哥伦比亚大学医学系);Department of Biomedical Data Science, Stanford School of Medicine(斯坦福医学院生物医学数据科学系);University of California, Berkeley(加州大学伯克利分校);University of California, San Francisco(加州大学旧金山分校);Department of Dermatology, Stanford School of Medicine(斯坦福医学院皮肤科)
Enhancing Large Language Models with Domain-specific Retrieval Augment Generation: A Case Study on Long-form Consumer Health Question Answering in Ophthalmology
Yale University(耶鲁大学);National Eye Institute, National Institutes of Health(美国国立卫生研究院国家眼科研究所);Singapore Eye Research Institute(新加坡眼科研究所);Singapore National Eye Centre(新加坡国家眼科中心);National Center for Biotechnology Information, National Library of Medicine, National Institutes of Health(美国国立卫生研究院国家医学图书馆国家生物技术信息中心);Rice University(莱斯大学);National University of Singapore(新加坡国立大学)
Performance of large language models in numerical vs. semantic medical knowledge: Benchmarking on evidence-based Q&As
Tel-Aviv University(特拉维夫大学);Kahun Medical Ltd(Kahun医疗有限公司);Hebrew University of Jerusalem(耶路撒冷希伯来大学);Leumit Health Services(Leumit医疗服务机构);Bar-Ilan University(巴伊兰大学);Kaplan Medical Center(卡普兰医疗中心);Brigham and Women’s Hospital(布莱根妇女医院);Harvard Medical School(哈佛医学院)
Deterministic Hallucination Detection in Medical VQA via Confidence-Evidence Bayesian Gain
Department of Electrical Engineering, Stanford University, CA, USA(电气工程系,斯坦福大学);Department of Biology, Stanford University, CA, USA(生物学系,斯坦福大学);Division of Cardiology, Department of Medicine, Stanford University, CA, USA(心脏病学部,医学系,斯坦福大学);Department of Biomedical Data Science, Stanford University, CA, USA(生物医学数据科学系,斯坦福大学);Department of Computer Science, Stanford University, CA, USA(计算机科学系,斯坦福大学);Department of Psychiatry and Behavioral Sciences, Stanford University, CA, USA(精神病学与行为科学系,斯坦福大学)
Faithful or Just Plausible? Evaluating the Faithfulness of Closed-Source LLMs in Medical Reasoning
Stratified Precision;Harvard Medical School(哈佛医学院);Imperial College London(帝国理工学院伦敦分校);National Health Service(国家健康服务系统);Ipsen France(Ipsen法国);University College London(伦敦大学学院)
Performance of ChatGPT-3.5 and GPT-4 on the United States Medical Licensing Examination With and Without Distractions
Magee-Womens Research Institute(马吉妇女研究所);University of Pittsburgh School of Medicine(匹兹堡大学医学院);Chaim Sheba Medical Center(哈伊姆·谢巴医疗中心);Sackler Faculty of Medicine, Tel Aviv University(特拉维夫大学萨克勒医学院);Ariel University(阿里尔大学)
LLM Factoscope: Uncovering LLMs' Factual Discernment through Inner States Analysis
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院)
From Physician Expertise to Clinical Agents: Preserving, Standardizing, and Scaling Physicians' Medical Expertise with Lightweight LLM
the School of Chinese Medicine, the Beijing University of Chinese Medicine, Beijing, China(中国中医药大学中医学院,北京中医药大学,北京,中国);the School of Pharmacy, Nanjing University of Chinese Medicine, Nanjing, China(南京中医药大学药学院,南京,中国);the Infectious disease department, Dongfang Hospital, Beijing University of Chinese Medicine, Beijing, China(北京中医药大学东四医院感染科,北京,中国);the Gulou Hospital of Traditional Chinese Medicine of Beijing,Beijing, China(北京中医药大学附属鼓楼中医医院,北京,中国);the Department of Computer Science, Johns Hopkins University, Baltimore, USA(约翰霍普金斯大学计算机科学系,美国马里兰州巴尔的摩市);the Department of Education, Dongzhimen Hospital, Beijing University of Chinese Medicine, Beijing, China(北京中医药大学东直门医院教育部,北京,中国);the Department of Pediatrics, Wangjing Hospital, China Academy of Chinese Medical Sciences, Beijing, China(中国医学科学院北京协和医院儿科部,北京,中国);the School of Information Engineering, Huzhou University, Huzhou, China(湖州大学信息工程学院,湖州,中国);Research Center for Scientific Data Hub, Zhejiang Lab, Hangzhou, China(浙江省实验室科学数据中心研究中心,杭州,中国);the Frontier Basic Research Center, Zhejiang Lab, Hangzhou, China(浙江省实验室前沿基础研究中心,杭州,中国);the Research Center for High Efficiency Computing Infrastructure, Zhejiang Lab, Hangzhou, China(浙江省实验室高效计算基础设施研究中心,杭州,中国)