This paper studies the performance of large language models (LLMs), particularly regarding demographic fairness, in solving real-world healthcare tasks. We evaluate state-of-the-art LLMs with three prevalent learning frameworks across six diverse healthcare tasks and find significant challenges in applying LLMs to real-world healthcare tasks and persistent fairness issues across demographic groups. We also find that explicitly providing demographic information yields mixed results, while LLM's ability to infer such details raises concerns about biased health predictions. Utilizing LLMs as autonomous agents with access to up-to-date guidelines does not guarantee performance improvement. We believe these findings reveal the critical limitations of LLMs in healthcare fairness and the urgent need for specialized research in this area.
Cross-Care: Assessing the Healthcare Implications of Pre-training Data on Language Model Bias
Harvard（哈佛大学）；Mass General Brigham（麻省总医院布里格姆医疗系统）；MIT（麻省理工学院）；Universitat Pompeu Fabra（庞培法布拉大学）；Aarhus University（奥胡斯大学）；University of North Carolina（北卡罗来纳大学）；Technical University of Munich（慕尼黑工业大学）；University of Zurich（苏黎世大学）；University of St. Gallen（圣加仑大学）；Maastricht University（马斯特里赫特大学）；Beth Israel Deaconess Medical Center（贝斯以色列女执事医疗中心）
Unveiling the Landscape of Clinical Depression Assessment: From Behavioral Signatures to Psychiatric Reasoning
School of Computer Science and Engineering, Central South University（中南大学计算机科学与工程学院）；CoAI Group, DCST, IAI, BNRIST, Tsinghua University（清华大学）；University of International Relations（国际关系大学）；Central China Normal University（华中师范大学）；Lingxin AI（灵心AI）；Yuquan Hospital, Tsinghua University（清华大学友谊医院）
The Future of Intelligent Healthcare: A Systematic Analysis and Discussion on the Integration and Impact of Robots Using Large Language Models for Healthcare
Deep Learning and Machine Learning, Advancing Big Data Analytics and Management: Unveiling AI's Potential Through Tools, Techniques, and Applications
National Taiwan Normal University（国立台湾师范大学）；Indiana University（印第安纳大学）；University of Hawaii（夏威夷大学）；Kyoto University（京都大学）；Xi’an Jiaotong-Liverpool University（西安交通大学利物浦大学）；Georgia Institute of Technology（佐治亚理工学院）；Rutgers University（罗格斯大学）；Purdue University（普渡大学）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；Emory University（埃默里大学）；Sichuan University（四川大学）
Seeds of Stereotypes: A Large-Scale Textual Analysis of Race and Gender Associations with Diseases in Online Sources
Aarhus University（奥胡斯大学）；Massachusetts Institute of Technology（麻省理工学院）；University of Alberta（阿尔伯塔大学）；University of Manitoba（曼尼托巴大学）；Mbarara University of Science and Technology（姆巴拉拉科技大学）；Fundacion Arturo Lopez Perez（阿图罗·洛佩斯·佩雷斯基金会）；Mass General Brigham（麻省总医院布里格姆）；Harvard Medical School（哈佛医学院）；Dana-Farber Cancer Institute（丹娜-法伯癌症研究所）；Emory University School of Medicine（埃默里大学医学院）；Beth Israel Deaconess Medical Center（贝斯以色列女执事医疗中心）；Harvard T.H. Chan School of Public Health（哈佛大学陈曾熙公共卫生学院）
本文通过分析 Arxiv、Wikipedia 和 Common Crawl 中18种疾病与种族、性别术语的共现，并对比患病率与 GPT-4 输出，揭示在线文本及 LLM 预训练数据中的人口统计学偏见。
Moving Beyond Medical Exams: A Clinician-Annotated Fairness Dataset of Real-World Tasks and Ambiguity in Mental Healthcare
Stanford University（斯坦福大学）；University of Colorado（科罗拉多大学）；Northwestern University（西北大学）；University of Wisconsin（威斯康星大学）；Yale School of Medicine（耶鲁医学院）；University of Chicago（芝加哥大学）；Ohio State University（俄亥俄州立大学）
The Voice of Equity: A Systematic Evaluation of Bias Mitigation Techniques for Speech-Based Cognitive Impairment Detection Across Architectures and Demographics
Columbia University Irving Medical Center（哥伦比亚大学欧文医学中心）；Columbia University School of Nursing（哥伦比亚大学护理学院）；Columbia University（哥伦比亚大学）
VITAL: A New Dataset for Benchmarking Pluralistic Alignment in Healthcare
School of Computing and Information System, the University of Melbourne, Australia（墨尔本大学计算与信息学院）；School of Computing, FSE, Macquarie University, Australia（麦考瑞大学计算学院）
