Large language models (LLMs) are increasingly embedded in healthcare workflows for documentation, education, and clinical decision support. However, these systems are trained on large text corpora that encode existing biases, including sex disparities in diagnosis and treatment, raising concerns that such patterns may be reproduced or amplified. We systematically examined whether contemporary LLMs exhibit sex-specific biases in clinical reasoning and how model configuration influences these behaviours. We conducted three experiments using 50 clinician-authored vignettes spanning 44 specialties in which sex was non-informative to the initial diagnostic pathway. Four general-purpose LLMs (ChatGPT (gpt-4o-mini), Claude 3.7 Sonnet, Gemini 2.0 Flash and DeepSeekchat). All models demonstrated significant sex-assignment skew, with predicted sex differing by model. At temperature 0.5, ChatGPT assigned female sex in 70% of cases (95% CI 0.66-0.75), DeepSeek in 61% (0.57-0.65) and Claude in 59% (0.55-0.63), whereas Gemini showed a male skew, assigning a female sex in 36% of cases (0.32-0.41). Contemporary LLMs exhibit stable, model-specific sex biases in clinical reasoning. Permitting abstention reduces explicit labelling but does not eliminate downstream diagnostic differences. Safe clinical integration requires conservative and documented configuration, specialty-level clinical data auditing, and continued human oversight when deploying general-purpose models in healthcare settings.
Evaluating Large Language Models' Responses to Sexual and Reproductive Health Queries in Nepali
Nepal Applied Mathematics and Informatics Institute for research（尼泊尔应用数学与信息学研究所）；Partnership for Sustainable Development Nepal（尼泊尔可持续发展伙伴关系）；Visible Impact（可见影响）
Calibrated? Not for Everyone: How Sexual Orientation and Religious Markers Distort LLM Accuracy and Confidence in Medical QA
Department of Medical Informatics, Amsterdam University Medical Center, University of Amsterdam, Amsterdam, The Netherlands（阿姆斯特丹大学医学信息学系，阿姆斯特丹大学医学中心，阿姆斯特丹，荷兰）；Amsterdam Public Health, Methodology, Amsterdam, The Netherlands（阿姆斯特丹公共卫生，方法学，阿姆斯特丹，荷兰）
University of Massachusetts, Amherst（马萨诸塞大学阿默斯特分校）；Emory University（埃默里大学）；University of Minnesota（明尼苏达大学）；University of Massachusetts, Lowell（马萨诸塞大学洛厄尔分校）；UMass Chan Medical School（UMass Chan医学学院）
Bridging the Knowledge-Action Gap by Evaluating LLMs in Dynamic Dental Clinical Scenarios
The Second Affiliated Hospital of Harbin Medical University（哈尔滨医科大学附属第二医院）；Peking University School and Hospital of Stomatology（北京大学口腔医学院（口腔医院））；Medlinker Intelligent and Digital Technology Co., Ltd（医联智数科技有限公司）；The University of Queensland（昆士兰大学）；Beijing Xicheng District Health Care Hospital for Mothers and Children（北京市西城区妇幼保健院）
MedFactEval and MedAgentBrief: A Framework and Workflow for Generating and Evaluating Factual Clinical Summaries
Center for Biomedical Informatics Research, Stanford University（生物医学信息学研究所在斯坦福大学）；Department of Biomedical Data Science, Stanford University（生物医学数据科学系，斯坦福大学）；Stanford Health Care（斯坦福健康中心）；Department of Anesthesiology and Pain Medicine, Stanford Medicine（麻醉与疼痛医学系，斯坦福医学院）；Department of Radiology, Stanford University（放射科，斯坦福大学）；Stanford Clinical Excellence Research Center, Stanford University（斯坦福临床卓越研究中心，斯坦福大学）；Department of Medicine, Stanford University（医学系，斯坦福大学）
FactEHR: A Dataset for Evaluating Factuality in Clinical Notes Using LLMs
Khoury College of Computer Sciences, Northeastern University（东北大学克劳尔计算机科学学院）；Center for Biomedical Informatics Research, Stanford University（斯坦福大学生物医学信息学研究中心）；Department of Biomedical Data Science, Stanford School of Medicine（斯坦福医学院生物医学数据科学系）；Stanford Health Care（斯坦福健康系统）；Department of Medicine, Stanford School of Medicine（斯坦福医学院医学系）；Clinical Excellence Research Center, Stanford School of Medicine（斯坦福医学院临床卓越研究中心）；Department of Anesthesiology, Perioperative & Pain Medicine, Stanford School of Medicine（斯坦福医学院麻醉学、手术及疼痛医学系）；Department of Dermatology, Stanford School of Medicine（斯坦福医学院皮肤病学系）；Technology and Digital Solutions, Stanford Health Care（斯坦福健康系统技术与数字解决方案）
SemioLLM: Evaluating Large Language Models for Diagnostic Reasoning from Unstructured Clinical Narratives in Epilepsy
University of Tübingen（蒂宾根大学）；Technical University of Munich（慕尼黑工业大学）；University Clinic Tübingen（蒂宾根大学医院）；Hertie Institute for Clinical Brain Research（赫蒂临床脑研究所）；Excellence Cluster Machine Learning, Tübingen University（蒂宾根大学机器学习卓越集群）；Hertie Institute for AI in Brain Health (Hertie AI)（赫蒂人工智能脑健康研究所）
Evaluating Large Language Model Raters for German Open-Response Clinical Questions: A Physician-Annotated Benchmark Study of Agreement, Evaluator Bias, and Abstention
University of Luebeck（吕贝克大学）；University of Tübingen（图宾根大学）；University Hospital Schleswig-Holstein（石勒苏益格-荷尔斯泰因大学医院）；University Hospital Würzburg（维尔茨堡大学医院）；Charité – Universitätsmedizin Berlin（柏林夏里特医学院）；Genie Enterprise Deutschland GmbH
A rubric landscape for evaluating clinical reasoning in large language models: what exists, what is missing, and what needs to be combined
DRIVE-Health CDT（DRIVE-Health 博士训练中心）；Department of Biostatistics and Health Informatics（生物统计学与健康信息学系）；Institute of Psychiatry, Psychology and Neuroscience（精神病学、心理学与神经科学研究所）；King’s College London（伦敦国王学院）；Cleveland Clinic London（伦敦克利夫兰诊所）
Evaluating Clinical Competencies of Large Language Models with a General Practice Benchmark
The Sixth Affiliated Hospital of Sun Yat-sen University（中山大学第六附属医院）；Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ)（广东省人工智能与数字经济发展实验室（深圳））；Xinyi People’s Hospital（新一人民医院）；The Fifth Affiliated Hospital of Sun Yat-sen University（中山大学第五附属医院）；School of Public Health of Sun Yat-sen University（中山大学公共卫生学院）
SUP-MIMIC: A Multi-Task Clinical Diagnosis Benchmark for Evaluating LLMs' Robustness to Contradictory Evidence
Beijing University of Technology（北京工业大学）；Beijing Institute of Technology（北京理工大学）；China-Japan Friendship Hospital（中日友好医院）；Dongbei University of Finance and Economics（东北财经大学）
LLAMADRS: Evaluating Open-Source LLMs on Real Clinical Interviews--To Reason or Not to Reason?
Carnegie Mellon University, School of Computer Science（卡内基梅隆大学计算机学院）；University of Kansas, Department of Psychology（堪萨斯大学心理学系）；McLean Hospital, Harvard Medical School（麦肯纳医院哈佛医学院）
PsychBench: A comprehensive and professional benchmark for evaluating the performance of LLM-assisted psychiatric clinical practice
Shanghai Jiao Tong University（上海交通大学）；Beijing Anding Hospital, Capital Medical University（首都医科大学附属北京安定医院）；Capital Medical University（首都医科大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）
A Decade-Scale Benchmark Evaluating LLMs' Clinical Practice Guidelines Detection and Adherence in Multi-turn Conversations
Microsoft Research Asia（微软亚洲研究院）；Hong Kong University of Science and Technology（香港科技大学）；Peking University（北京大学）；Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
Evaluating Large Language Models on the Spanish Medical Intern Resident (MIR) Examination 2024/2025:A Comparative Analysis of Clinical Reasoning and Knowledge Application
SiMing-Bench: Evaluating Procedural Correctness from Continuous Interactions in Clinical Skill Videos
School of Artificial Intelligence, Wuhan University（武汉大学人工智能学院）；Center for Language and Information Research, Wuhan University（武汉大学语言与信息研究中心）；Southwest Jiaotong University（西南交通大学）；MBZUAI（穆罕默德·本·扎耶德人工智能大学）；The University of Manchester（曼彻斯特大学）；Zhongnan Hospital of Wuhan University（武汉大学中南医院）
Same Patient, Different Words, Different Diagnosis? Evaluating Semantic Stability in Clinical LLMs
Department of Computer Science and Engineering, College of Engineering, Qatar University（卡塔尔大学计算机科学与工程系）；College of Science and Engineering, Hamad Bin Khalifa University (HBKU)（哈马德·本·卡伊夫大学（HBKU）理学院）；Primary Health Care Corporation (PHCC)（初级卫生保健公司）；Birmingham City University（伯明翰城市大学）
K-Bench: a clinically calibrated benchmark for evaluating large language models in high-risk mental health conversations
University of Roehampton（罗汉普顿大学）；Kivira Health；University of Hertfordshire（赫特福德大学）；University of Surrey（萨里大学）；University of Bedfordshire（贝德福德大学）；Tavistock Relationships（塔维斯托克关系中心）；InsideOut
ClinBench-HPB: A Clinical Benchmark for Evaluating LLMs in Hepato-Pancreato-Biliary Diseases
The Hong Kong Polytechnic University（香港理工大学）；Shenzhen Institutes of Advanced Technology, CAS（深圳先进技术研究院）；Zhujiang Hospital, Southern Medical University（南方医科大学珠江医院）；The Key Laboratory of Biomedical Imaging Science and System, CAS（生物医学成像科学与系统重点实验室）
Evaluating Multi-Turn Multimodal Diagnostic Reasoning on Challenging Real-World Clinical Cases
Center for Biomedical Data Science, Duke-NUS Medical School（生物医学数据科学中心，杜克 - 新加坡国立大学医学院）；Duke-NUS AI + Medical Sciences Initiative, Duke-NUS Medical School（杜克 - 新加坡国立大学人工智能与医学科学计划，杜克 - 新加坡国立大学医学院）；Department of Population Health Sciences, Weill Cornell Medicine（人口健康科学系，威尔康奈尔医学院）；System Engineering, College of Engineering, Cornell University（系统工程，康奈尔大学工程学院）；Graduate School of Frontier Sciences, The University of Tokyo（前沿科学研究生院，东京大学）；RIKEN Center for Advanced Intelligence Project（理化学研究所先进智能项目中心）；Department of Biostatistics and Bioinformatics, Duke University（生物统计学与生物信息学系，杜克大学）；Perelman School of Medicine, University of Pennsylvania（佩雷尔曼医学院，宾夕法尼亚大学）；School of Clinical Medicine, University of Cambridge（临床医学学院，剑桥大学）；Hospital of the University of Pennsylvania（宾夕法尼亚大学医院）；Children’s Hospital of Philadelphia (CHOP)（费城儿童医院）；Department of Anatomic Pathology and Laboratory Medicine, Hospital of the University of Pennsylvania（解剖病理学与检验医学系，宾夕法尼亚大学医院）；Department of Pathology and Laboratory Medicine, University of California, San Francisco（病理学与检验医学系，加州大学旧金山分校）；Language Technologies Institute, Carnegie Mellon University（语言技术研究所，卡内基梅隆大学）
Evaluating the Impact of Lab Test Results on Large Language Models Generated Differential Diagnoses from Clinical Case Vignettes
Florida State University（佛罗里达州立大学）；National Center for Biotechnology Information, National Library of Medicine（美国国家医学图书馆国家生物技术信息中心）；Emory University（埃默里大学）；Morsani College of Medicine, University of South Florida（南佛罗里达大学莫萨尼医学院）
ClinicRealm: Re-evaluating Large Language Models with Conventional Machine Learning for Non-Generative Clinical Prediction Tasks
School of Artificial Intelligence, Beihang University（北京航空航天大学人工智能学院）；Centre for Medical Informatics, The University of Edinburgh（爱丁堡大学医学信息学中心）；Health Data Research UK（英国健康数据研究所）；School of Computer Science, Peking University（北京大学计算机学院）；National Engineering Research Center for Software Engineering, Peking University（北京大学软件工程国家工程研究中心）；ETH Zurich（苏黎世联邦理工学院）；School of Computing and Data Science, The University of Hong Kong（香港大学计算与数据科学学院）
TherapyGym: Evaluating and Aligning Clinical Fidelity and Safety in Therapy Chatbots
Department of Computer Science, Stanford University, Stanford, CA, USA（计算机科学系，斯坦福大学，斯坦福，加州，美国）；Department of Psychiatry and Behavioral Sciences, Stanford University, Stanford, CA, USA（精神病学与行为科学系，斯坦福大学，斯坦福，加州，美国）；Department of Biomedical Data Science, Stanford University, Stanford, CA, USA（生物医学数据科学系，斯坦福大学，斯坦福，加州，美国）；University of California, Berkeley, Berkeley, CA, USA（加州大学伯克利分校，伯克利，加州，美国）；The University of Hong Kong, Hong Kong（香港大学，香港）
ECG-Expert-QA: A Benchmark for Evaluating Medical Large Language Models in Heart Disease Diagnosis
Zhejiang University School of Medicine（浙江大学医学院）；The First Affiliated Hospital（第一附属医院）；Liangzhu Laboratory（良渚实验室）；Shandong Jianzhu University（山东建筑大学）；FUXI AI Lab（伏羲人工智能实验室）；Beijing Normal University（北京师范大学）；Southern University of Science and Technology（南方科技大学）；China University of Geosciences (Wuhan)（中国地质大学（武汉））；Hangzhou Baorun Biotechnology Co., Ltd.（杭州宝润生物科技有限公司）
