Summarizing clinical text is crucial in health decision-support and clinical research. Large language models (LLMs) have shown the potential to generate accurate clinical text summaries, but still struggle with issues regarding grounding and evaluation, especially in safety-critical domains such as health. Holistically evaluating text summaries is challenging because they may contain unsubstantiated information. Here, we explore a general mitigation framework using Attribute Structuring (AS), which structures the summary evaluation process. It decomposes the evaluation process into a grounded procedure that uses an LLM for relatively simple structuring and scoring tasks, rather than the full task of holistic summary evaluation. Experiments show that AS consistently improves the correspondence between human annotations and automated metrics in clinical text summarization. Additionally, AS yields interpretations in the form of a short text span corresponding to each output, which enables efficient human auditing, paving the way towards trustworthy evaluation of clinical information in resource-constrained scenarios. We release our code, prompts, and an open-source benchmark at https://github.com/microsoft/attribute-structuring.
East China University of Science and Technology（华东理工大学）；Ruijin Hospital, School of Medicine, Shanghai Jiao Tong University（上海交通大学医学院附属瑞金医院）
Clinical text structuring is a critical and fundamental task for clinical research. Traditional methods such as taskspecific end-to-end models and pipeline models usually suffer from the lack of dataset and error propagation. In this paper, we present a question answering based clinical text structuring (QA-CTS) task to unify different specific tasks and make dataset shareable. A novel model that aims to introduce domain-specific features (e.g., clinical named entity information) into pre-trained language model is also proposed for QA-CTS task. Experimental results on Chinese pathology reports…
EHRNote-ChatQA: A Benchmark for Evidence-Grounded Multi-Turn Clinical Question Answering over Longitudinal Discharge Summaries
KAIST（韩国科学技术院）；Seoul National University（首尔大学）；Seoul National University Bundang Hospital（首尔大学盆唐医院）；SAIHST, Sungkyunkwan University（成均馆大学）；Yonsei University College of Medicine（延世大学医学院）；Gangnam Severance Hospital（江南塞弗伦斯医院）；Severance Hospital（塞弗伦斯医院）；Seoul Medical Center（首尔医疗中心）；Seoul National University Hospital（首尔大学医院）；National Cancer Center（国立癌症中心）；Icahn School of Medicine at Mount Sinai（西奈山伊坎医学院）；Samsung Medical Center（三星医疗中心）
DistillNote: Toward a Functional Evaluation Framework of LLM-Generated Clinical Note Summaries
Department of Medical Informatics, Amsterdam UMC, University of Amsterdam（医学信息学系，阿姆斯特丹大学）；Amsterdam Public Health, Methodology（阿姆斯特丹公共健康与方法学）；Institute of Mathematics and Statistics, University of São Paulo（数学与统计学研究所，圣保罗大学）
MedFactEval and MedAgentBrief: A Framework and Workflow for Generating and Evaluating Factual Clinical Summaries
Center for Biomedical Informatics Research, Stanford University（生物医学信息学研究所在斯坦福大学）；Department of Biomedical Data Science, Stanford University（生物医学数据科学系，斯坦福大学）；Stanford Health Care（斯坦福健康中心）；Department of Anesthesiology and Pain Medicine, Stanford Medicine（麻醉与疼痛医学系，斯坦福医学院）；Department of Radiology, Stanford University（放射科，斯坦福大学）；Stanford Clinical Excellence Research Center, Stanford University（斯坦福临床卓越研究中心，斯坦福大学）；Department of Medicine, Stanford University（医学系，斯坦福大学）
Ten Headache Specialists versus Artificial Intelligence for Clinical Literature Summarization: A Critical Evaluation and Comparison
Stanford University Palo Alto CA USA（斯坦福大学）；Department of Neurology Mayo Clinic Rochester MN USA（梅奥诊所神经科）；Department of Neurology Dalhousie University Halifax Canada（达尔豪斯大学神经科）；Jefferson Headache Center Department of Neurology Thomas Jefferson University PA USA（泰勒大学神经科）；Beth Israel Deaconess Medical Center Boston MA USA（贝斯以色列医疗中心）；Department of Neurology University of Florida Gainesville FL USA（佛罗里达大学神经科）；University of Colorado School of Medicine Department of Pediatrics Division of Child Neurology Aurora CO USA（科罗拉多医学院儿科部儿童神经科）；Department of Medicine Mount Sinai Hospital Icahn School of Medicine at Mount Sinai New York NY USA（西奈医院医学部）；Department of Neurology Mayo Clinic Scottsdale AZ USA（梅奥诊所Scottsdale分部）；Harvard Medical School Boston MA USA（哈佛医学院）；Department of Neurology Mount Sinai Hospital Icahn School of Medicine at Mount Sinai New York NY USA（西奈医院神经科）
Comparing Large Language Models and Traditional Machine Translation Tools for Translating Medical Consultation Summaries: A Pilot Study
Faculty of Information Technology, Monash University（信息技术学院，莫纳什大学）；Faculty of Medicine, Nursing and Health Sciences, Monash University（医学、护理与健康科学学院，莫纳什大学）；Supportive and Palliative Care Unit, Monash Health（支持与姑息护理单元，莫纳什健康）
The use of large language models to enhance cancer clinical trial educational materials
Massachusetts Institute of Technology（麻省理工学院）；Technical University of Munich（慕尼黑工业大学）；Mass General Brigham（麻省总医院布里格姆医疗系统）；Harvard Medical School（哈佛医学院）；Brigham and Women’s Hospital（布莱根妇女医院）；Dana-Farber Cancer Institute（丹娜-法伯癌症研究所）；Centaur Labs（半人马实验室）；Digidence；Loyola University Chicago（芝加哥洛约拉大学）
MedPAO: A Protocol-Driven Agent for Structuring Medical Reports
Department of Data Science and AI, IIT Madras, India（数据科学与人工智能系，印度理工学院马德拉斯学院）；Department of Engineering Design, IIT Madras, India（工程设计系，印度理工学院马德拉斯学院）；LoveForm Health Technologies, India（LoveForm健康科技公司，印度）；Department of Radiology and Imaging Sciences, Sri Ramachandra Institute of Higher Education and Research, India（放射学与成像科学系， Sri Ramachandra高等教育与研究学院，印度）；Department of Neuro and Interventional Radiology, Sri Ramachandra Institute of Higher Education and Research, India（神经放射学与介入放射学系，Sri Ramachandra高等教育与研究学院，印度）
High-Throughput Phenotyping of Clinical Text Using Large Language Models
Dept. of Neurology \& Rehabilitation University of Illinois at Chicago Chicago, IL USA；Engineering Program Missouri State University Springfield, MO USA
Phase 1 Implementation of LLM-generated Discharge Summaries showing high Adoption in a Dutch Academic Hospital
Department of Health Information Office, Data & Digitalization, University Medical Center（健康信息办公室、数据与数字化化大学医院）；Department of Trauma Surgery / Orthopedics, University Medical Center（创伤外科/骨科大学医院）；Department of Orthopaedic Surgery, Massachusetts General Hospital（骨科手术科马萨诸塞总医院）；Harvard Medical School Orthopaedic Trauma Initiative（哈佛医学院骨创伤倡议）；Department of Orthopaedic Surgery and Sports Medicine, Amsterdam UMC location University of Amsterdam（骨科手术与运动医学部门，阿姆斯特丹UMC地点大学）；Department of Orthopaedic Trauma, Flinders University Medical Centre（骨创伤部门，弗林德斯大学医疗中心）；Department of Otolaryngology – Head and Neck Surgery, University Medical Center（耳鼻喉科-头颈外科大学医院）；Department of Pediatrics, Pediatrics Nephrology, Beatrix Children’s Hospital, University Medical Center（儿科、儿科肾病学，比西特儿童医院，大学医院）
Feasibility of Structuring Stress Documentation Using an Ontology-Guided Large Language Model
College of Nursing, Seoul National University（首尔国立大学护理学院）；The Research Institute of Nursing Science, Seoul National University（首尔国立大学护理科学研究所）；Center of Human-Caring Nurse Leaders for the Future by the Brain Korea (BK 21) four project, College of Nursing, Seoul National University（首尔国立大学护理学院，韩国脑计划（BK21）第四阶段未来人文关怀护理领军人才中心）；College of Medicine, Seoul National University（首尔国立大学医学院）
MIMIC-\RNum{4}-Ext-22MCTS: A 22 Millions-Event Temporal Clinical Time-Series Dataset with Relative Timestamp for Risk Prediction
National Library of Medicine（国家医学图书馆）；AWS AI Labs（AWS人工智能实验室）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Stevens Institute of Technology（史蒂文斯理工学院）；Carnegie Mellon University（卡内基梅隆大学）
Clinical trial records are variable resources or the analysis of patients and diseases. Information extraction from free text such as eligibility criteria and summary of results and conclusions in clinical trials would better support computer-based eligibility query formulation and electronic patient screening. Previous research has focused on extracting information from eligibility criteria, with usually a single pair of medical entity and attribute, but seldom considering other kinds of free text with multiple entities, attributes and relations that are more complex for parsing. In this pape…
