Unstructured clinical text in EHRs contains crucial information for applications including decision support, trial matching, and retrospective research. Recent work has applied BERT-based models to clinical information extraction and text classification, given these models' state-of-the-art performance in other NLP domains. However, BERT is difficult to apply to clinical notes because it doesn't scale well to long sequences of text. In this work, we propose a novel fine-tuning approach called SnipBERT. Instead of using entire notes, SnipBERT identifies crucial snippets and then feeds them into a truncated BERT-based model in a hierarchical manner. Empirically, SnipBERT not only has significant predictive performance gain across three tasks but also provides improved interpretability, as the model can identify key pieces of text that led to its prediction.
Berliner Hochschule für Technik (BHT)（柏林工程应用技术大学）；Charité Berlin（柏林夏里特医学院）；Leibniz University Hannover（汉诺威莱布尼茨大学）
The use of deep neural models for diagnosis prediction from clinical text has shown promising results. However, in clinical practice such models must not only be accurate, but provide doctors with interpretable and helpful results. We introduce ProtoPatient, a novel method based on prototypical networks and label-wise attention with both of these abilities. ProtoPatient makes predictions based on parts of the text that are similar to prototypical patients - providing justifications that doctors understand. We evaluate the model on two publicly available clinical datasets and show that it outpe…
Clinically Grounded Agent-based Report Evaluation: An Interpretable Metric for Radiology Report Generation
New York University（纽约大学）；NYU Langone Health（纽约大学兰格医学中心）；Genentech（基因泰克）；NYU Grossman School of Medicine（纽约大学格罗斯曼医学院）；Wenzhou Medical University（温州医科大学）；Macau University of Science and Technology（澳门科学大学）
Antwerp University Hospital（安特卫普大学医院）；CLiPS Research Center, University of Antwerp（安特卫普大学 CLiPS 研究中心）
We have two main contributions in this work: 1. We explore the usage of a stacked denoising autoencoder, and a paragraph vector model to learn task-independent dense patient representations directly from clinical notes. We evaluate these representations by using them as features in multiple supervised setups, and compare their performance with those of sparse representations. 2. To understand and interpret the representations, we explore the best encoded features within the patient representations obtained from the autoencoder model. Further, we calculate the significance of the input features…
Parameter-Efficient Fine-Tuning of LLaMA for the Clinical Domain
School of Informatics, University of Edinburgh（爱丁堡大学信息学院）；Usher Institute, University of Edinburgh（爱丁堡大学厄舍研究所）；Allen Institute of AI（艾伦人工智能研究所）；Hebrew University of Jerusalem（耶路撒冷希伯来大学）；Edinburgh Futures Institute, University of Edinburgh（爱丁堡大学未来学院）；School of Literatures, Languages and Cultures, University of Edinburgh（爱丁堡大学文学、语言与文化学院）
Boston Children’s Hospital（波士顿儿童医院）；Harvard Medical School（哈佛医学院）
Large scale contextual representation models, such as BERT, have significantly advanced natural language processing (NLP) in recently years. However, in certain area like healthcare, accessing diverse large scale text data from multiple institutions is extremely challenging due to privacy and regulatory reasons. In this article, we show that it is possible to both pretrain and fine tune BERT models in a federated manner using clinical texts from different silos without moving the data.
Antwerp University Hospital（安特卫普大学医院）；University of Antwerp（安特卫普大学）
We have three contributions in this work: 1. We explore the utility of a stacked denoising autoencoder and a paragraph vector model to learn task-independent dense patient representations directly from clinical notes. To analyze if these representations are transferable across tasks, we evaluate them in multiple supervised setups to predict patient mortality, primary diagnostic and procedural category, and gender. We compare their performance with sparse representations obtained from a bag-of-words model. We observe that the learned generalized representations significantly outperform the spar…
A Modular Approach for Clinical SLMs Driven by Synthetic Data with Pre-Instruction Tuning, Model Merging, and Clinical-Tasks Alignment
Microsoft Healthcare & Life Sciences（微软医疗与生命科学）；Microsoft Research Montréal（微软研究院蒙特利尔）；Mila, Université de Montréal（蒙特利尔大学米拉研究所）；IKIM, University Hospital Essen（埃森大学医院人工智能医学研究所）；Cancer Research Center Cologne Essen（科隆埃森癌症研究中心）；German Cancer Consortium（德国癌症联盟）；TU Dortmund（多特蒙德工业大学）
Psychologically-Grounded Graph Modeling for Interpretable Depression Detection
Singapore Institute of Technology（新加坡科技学院）；Nanyang Technological University（南洋理工大学）；ELLIS Institute Finland（芬兰ELLIS研究所）；University of Turku（图尔库大学）；Princess Nourah bint Abdulrahman University（普里西拉努拉·本·阿卜杜勒拉赫曼大学）
NIHR Oxford Health Biomedical Research Centre（NIHR牛津健康生物医学研究中心）；University of Oxford（牛津大学）
Prompt learning is a new paradigm in the Natural Language Processing (NLP) field which has shown impressive performance on a number of natural language tasks with common benchmarking text datasets in full, few-shot, and zero-shot train-evaluation setups. Recently, it has even been observed that large but frozen pre-trained language models (PLMs) with prompt learning outperform smaller but fine-tuned models. However, as with many recent NLP trends, the performance of even the largest PLMs such as GPT-3 do not perform well on specialized domains (e.g. medical text), and the common practice to ac…
Auckland University of Technology（奥克兰理工大学）；Universidad Tecnológica de Bolívar（玻利瓦尔理工大学）
Accurate mortality prediction allows Intensive Care Units (ICUs) to adequately benchmark clinical practice and identify patients with unexpected outcomes. Traditionally, simple statistical models have been used to assess patient death risk, many times with sub-optimal performance. On the other hand deep learning holds promise to positively impact clinical practice by leveraging medical data to assist diagnosis and prediction, including mortality prediction. However, as the question of whether powerful Deep Learning models attend correlations backed by sound medical knowledge when generating pr…
Interpretable Symptom Vectors for Depression in a Large Language Model
Stanford University School of Medicine（斯坦福大学医学院）；Wu Tsai Neurosciences Institute, Stanford University（斯坦福大学吴蔡神经科学研究所）；Kaiser Permanente（凯撒医疗机构）；BigCommerce（BigCommerce公司）；Veterans Affairs Palo Alto Healthcare System（帕洛阿尔托退伍军人事务医疗系统）；Sierra Pacific Mental Illness, Research, Education, and Clinical Center (MIRECC)（塞拉太平洋精神疾病研究、教育和临床中心（MIRECC））
