Unsupervised domain adaptation (UDA) generally aligns the unlabeled target domain data to the distribution of the source domain to mitigate the distribution shift problem. The standard UDA requires sharing the source data with the target, having potential data privacy leaking risks. To protect the source data's privacy, we first propose to share the source feature distribution instead of the source data. However, sharing only the source feature distribution may still suffer from the membership inference attack who can infer an individual's membership by the black-box access to the source model. To resolve this privacy issue, we further study the under-explored problem of privacy-preserving domain adaptation and propose a method with a novel differential privacy training strategy to protect the source data privacy. We model the source feature distribution by Gaussian Mixture Models (GMMs) under the differential privacy setting and send it to the target client for adaptation. The target client resamples differentially private source features from GMMs and adapts on target data with several state-of-art UDA backbones. With our proposed method, the source data provider could avoid leaking source data privacy during domain adaptation as well as reserve the utility. To evaluate our proposed method's utility and privacy loss, we apply our model on a medical report disease label classification task using two noisy challenging clinical text datasets. The results show that our proposed method can preserve source data's privacy with a minor performance influence on the text classification task.
Preserving Privacy, Increasing Accessibility, and Reducing Cost: An On-Device Artificial Intelligence Model for Medical Transcription and Note Generation
Small Language Models for Privacy-Preserving Clinical Information Extraction in Low-Resource Languages
Student Research Committee Isfahan University of Medical Sciences（伊斯法罕医学科学大学学生研究委员会）；Department of Emergency Medicine Isfahan University of Medical Sciences（伊斯法罕医学科学大学急诊医学科）
The Ohio State University（俄亥俄州立大学）；Abigail Wexner Research Institute at Nationwide Children’s Hospital（全国儿童医院阿比盖尔·韦克斯纳研究所）
Unstructured clinical texts contain rich health-related information. To better utilize the knowledge buried in clinical texts, discovering synonyms for a medical query term has become an important task. Recent automatic synonym discovery methods leveraging raw text information have been developed. However, to preserve patient privacy and security, it is usually quite difficult to get access to large-scale raw clinical texts. In this paper, we study a new setting named synonym discovery on privacy-aware clinical data (i.e., medical terms extracted from the clinical texts and their aggregated co…
University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）；University of Massachusetts Lowell（马萨诸塞大学洛厄尔分校）
Deep Neural Network (DNN) models have been shown to have high empirical privacy leakages. Clinical language models (CLMs) trained on clinical data have been used to improve performance in biomedical natural language processing tasks. In this work, we investigate the risks of training-data leakage through white-box or black-box access to CLMs. We design and employ membership inference attacks to estimate the empirical privacy leaks for model architectures like BERT and GPT2. We show that membership inference attacks on CLMs lead to non-trivial privacy leakages of up to 7%. Our results show that…
Antwerp University Hospital（安特卫普大学医院）；CLiPS Research Center, University of Antwerp（安特卫普大学 CLiPS 研究中心）
We have two main contributions in this work: 1. We explore the usage of a stacked denoising autoencoder, and a paragraph vector model to learn task-independent dense patient representations directly from clinical notes. We evaluate these representations by using them as features in multiple supervised setups, and compare their performance with those of sparse representations. 2. To understand and interpret the representations, we explore the best encoded features within the patient representations obtained from the autoencoder model. Further, we calculate the significance of the input features…
Queensland University of Technology（昆士兰科技大学）；The Australian e-Health Research Centre, CSIRO（澳大利亚电子健康研究中心，联邦科学与工业研究组织）
This study investigates the use of unsupervised word embeddings and sequence features for sample representation in an active learning framework built to extract clinical concepts from clinical free text. The objective is to further reduce the manual annotation effort while achieving higher effectiveness compared to a set of baseline features. Unsupervised features are derived from skip-gram word embeddings and a sequence representation approach. The comparative performance of unsupervised features and baseline hand-crafted features in an active learning framework are investigated using a wide …
We present an unsupervised context-sensitive spelling correction method for clinical free-text that uses word and character n-gram embeddings. Our method generates misspelling replacement candidates and ranks them according to their semantic fit, by calculating a weighted cosine similarity between the vectorized representation of a candidate and the misspelling context. To tune the parameters of this model, we generate self-induced spelling error corpora. We perform our experiments for two languages. For English, we greatly outperform off-the-shelf spelling correction tools on a manually annot…
As patients' access to their doctors' clinical notes becomes common, translating professional, clinical jargon to layperson-understandable language is essential to improve patient-clinician communication. Such translation yields better clinical outcomes by enhancing patients' understanding of their own health conditions, and thus improving patients' involvement in their own care. Existing research has used dictionary-based word replacement or definition insertion to approach the need. However, these methods are limited by expert curation, which is hard to scale and has trouble generalizing to …
University of Memphis（孟菲斯大学）；Adobe Research（奥多比研究院）；Johns Hopkins University（约翰斯·霍普金斯大学）
Clinical notes in Electronic Health Records (EHR) present rich documented information of patients to inference phenotype for disease diagnosis and study patient characteristics for cohort selection. Unsupervised user embedding aims to encode patients into fixed-length vectors without human supervisions. Medical concepts extracted from the clinical notes contain rich connections between patients and their clinical categories. However, existing unsupervised approaches of user embeddings from clinical notes do not explicitly incorporate medical concepts. In this study, we propose a concept-aware …
Hong Kong Baptist University（香港浸会大学）；Pangaea Data Limited（潘加亚数据有限公司）；Imperial College London（帝国理工学院）；Queen Mary University of London（伦敦玛丽女王大学）
Extracting phenotypes from clinical text has been shown to be useful for a variety of clinical use cases such as identifying patients with rare diseases. However, reasoning with numerical values remains challenging for phenotyping in clinical text, for example, temperature 102F representing Fever. Current state-of-the-art phenotyping models are able to detect general phenotypes, but perform poorly when they detect phenotypes requiring numerical reasoning. We present a novel unsupervised methodology leveraging external knowledge and contextualized word embeddings from ClinicalBERT for numerical…
DP-IPI: A Hybrid Differential Privacy Text Rewriting Mechanism for Indirect Personal Identifiers in Clinical Texts
Technical University of Berlin（柏林工业大学）；German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心（DFKI））；Technical University of Munich（慕尼黑工业大学）；Munich Center for Machine Learning（慕尼黑机器学习中心）
University of Ottawa（渥太华大学）；Dalhousie University（达尔豪斯大学）
In this study, we explored application of Word2Vec and Doc2Vec for sentiment analysis of clinical discharge summaries. We applied unsupervised learning since the data sets did not have sentiment annotations. Note that unsupervised learning is a more realistic scenario than supervised learning which requires an access to a training set of sentiment-annotated data. We aim to detect if there exists any underlying bias towards or against a certain disease. We used SentiWordNet to establish a gold sentiment standard for the data sets and evaluate performance of Word2Vec and Doc2Vec methods. We have…
De-identification is not enough: a comparison between de-identified and synthetic clinical notes
Department of Computer Science, University of Manitoba（曼尼托巴大学计算机科学系）；McWilliams School of Biomedical Informatics, The University of Texas Health Science Center at Houston（德克萨斯大学休斯顿健康科学中心麦克威廉姆斯生物医学信息学院）
IBM T. J. Watson Research Center（IBM T. J. 沃森研究中心）；IBM Almaden Research Center（IBM阿尔马登研究中心）
Large-scale clinical data is invaluable to driving many computational scientific advances today. However, understandable concerns regarding patient privacy hinder the open dissemination of such data and give rise to suboptimal siloed research. De-identification methods attempt to address these concerns but were shown to be susceptible to adversarial attacks. In this work, we focus on the vast amounts of unstructured natural language data stored in clinical notes and propose to automatically generate synthetic clinical notes that are more amenable to sharing using generative models trained on r…
Isfahan University of Technology（伊斯法罕理工大学）；Medical University of Vienna（维也纳医科大学）；University of Isfahan（伊斯法罕大学）
Clinical notes containing valuable patient information are written by different health care providers with various scientific levels and writing styles. It might be helpful for clinicians and researchers to understand what information is essential when dealing with extensive electronic medical records. Entities recognizing and mapping them to standard terminologies is crucial in reducing ambiguity in processing clinical notes. Although named entity recognition and entity linking are critical steps in clinical natural language processing, they can also result in the production of repetitive and…
Coding-Free and Privacy-Preserving Agentic Framework for Data-Driven Clinical Research
Infmedix, Co., Ltd.（Infmedix公司）；Department of Transdisciplinary Studies, Seoul National University（首尔国立大学跨学科研究系）；Healthcare AI Research Institute, Seoul National University Hospital（首尔国立大学医院医疗人工智能研究所）；Department of Medicine, Seoul National University College of Medicine（首尔国立大学医学院医学系）；Department of Transdisciplinary Medicine, Seoul National University Hospital（首尔国立大学医院跨学科医学系）；Department of Radiology, Massachusetts General Hospital and Harvard Medical School（麻省总医院放射科及哈佛医学院）
Self-Prompting Small Language Models for Privacy-Sensitive Clinical Information Extraction
McWilliams School of Biomedical Informatics, The University of Texas Health Science Center at Houston（德克萨斯大学健康科学中心休斯顿分校麦克威廉斯生物医学信息学学院）；School of Public Health, The University of Texas Health Science Center at Houston（德克萨斯大学健康科学中心休斯顿分校公共卫生学院）；School of Dentistry, The University of Texas Health Science Center at Houston（德克萨斯大学健康科学中心休斯顿分校牙科学院）；Willamette Dental and Skourtes Institute（威廉特牙科与斯库尔特斯研究所）
DeIDClinic: A Risk-Aware Pseudonymization Framework for Clinical Text De-identification and Re-identification Risk Assessment
University of Manchester（曼彻斯特大学）；Leiden Institute of Advanced Computer Science (LIACS)（莱顿先进计算机科学研究所）；Leiden University（莱顿大学）；Biomedical Data Sciences, Leiden University Medical Center（莱顿大学医学中心生物医学数据科学）
Memorial Sloan Kettering Cancer Center（纪念斯隆凯特琳癌症中心）；Weill Cornell Medicine（威尔康奈尔医学院）；ETH Zürich（苏黎世联邦理工学院）；University Hospital Zürich（苏黎世大学医院）；Swiss Institute for Bioinformatics（瑞士生物信息学研究所）；University Carlos III in Madrid（马德里卡洛斯三世大学）；Harvard University（哈佛大学）；University of Basel（巴塞尔大学）；University Hospital Basel（巴塞尔大学医院）；Swiss Data Science Center（瑞士数据科学中心）
The recent adoption of Electronic Health Records (EHRs) by health care providers has introduced an important source of data that provides detailed and highly specific insights into patient phenotypes over large cohorts. These datasets, in combination with machine learning and statistical approaches, generate new opportunities for research and clinical care. However, many methods require the patient representations to be in structured formats, while the information in the EHR is often locked in unstructured texts designed for human readability. In this work, we develop the methodology to automa…
Selective Token-Level Cryptographic Redaction for Privacy-Preserving Clinical Deployment of Large Language Models
MedVisAI Lab, Singapore（新加坡MedVisAI实验室）；Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, China（中国上海交通大学医学院瑞金医院）；Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore（新加坡南洋理工大学Lee Kong Chian医学院）
Privacy-Preserving Local Language Models for Longitudinal Data Retrieval in Chronic Dermatologic Disease: Implementation in Pemphigus Patients
Division of Systems Medicine, Department of Metabolism, Digestion and Reproduction, Imperial College London（系统医学系，代谢、消化与生殖部，帝国理工学院伦敦分校）；Department of Dermatology and Venereology, Istanbul Research and Training Hospital（皮肤科与性病科，伊斯坦布尔研究与培训医院）；Department of Dermatology and Venereology, Istanbul Medeniyet University（皮肤科与性病科，伊斯坦布尔梅德尼yet大学）；Department of Dermatology and Venereology, Istanbul Medicana Atakoy Hospital（皮肤科与性病科，伊斯坦布尔Medicana阿塔科伊医院）
Comparative analysis of privacy-preserving open-source LLMs regarding extraction of diagnostic information from clinical CMR imaging reports
The D-Lab, Department of Precision Medicine, GROW - Research Institute for Oncology and Reproduction, Maastricht University（D实验室，精准医学系，GROW癌症与生殖研究研究所，马斯特里赫特大学）；Division of Cardiovascular Imaging, Department of Cardiology I, University Hospital Münster（心血管影像科，心内科第一部门，穆恩斯特大学医院）
Korea Advanced Institute of Science & Technology (KAIST)（韩国科学技术院）；NAVER CLOVA AI Lab（NAVER CLOVA AI实验室）
We propose Medical Entity Definition-based Sentence Embedding (MED-SE), a novel unsupervised contrastive learning framework designed for clinical texts, which exploits the definitions of medical entities. To this end, we conduct an extensive analysis of multiple sentence embedding techniques in clinical semantic textual similarity (STS) settings. In the entity-centric setting that we have designed, MED-SE achieves significantly better performance, while the existing unsupervised methods including SimCSE show degraded performance. Our experiments elucidate the inherent discrepancies between the…
Boston Children’s Hospital（波士顿儿童医院）；Harvard Medical School（哈佛医学院）
Large scale contextual representation models, such as BERT, have significantly advanced natural language processing (NLP) in recently years. However, in certain area like healthcare, accessing diverse large scale text data from multiple institutions is extremely challenging due to privacy and regulatory reasons. In this article, we show that it is possible to both pretrain and fine tune BERT models in a federated manner using clinical texts from different silos without moving the data.
University of Antwerp（安特卫普大学）；Antwerp University Hospital（安特卫普大学医院）
Clinical NLP has an immense potential in contributing to how clinical practice will be revolutionized by the advent of large scale processing of clinical records. However, this potential has remained largely untapped due to slow progress primarily caused by strict data access policies for researchers. In this paper, we discuss the concern for privacy and the measures it entails. We also suggest sources of less sensitive data. Finally, we draw attention to biases that can compromise the validity of empirical research and lead to socially harmful applications.
Retrieval-Reasoning Large Language Model-based Synthetic Clinical Trial Generation
Institute for Clarity in Documentation（清晰文档研究所）；Inria Paris-Rocquencourt（巴黎- Rocquencourt 国家信息与自动化研究所）；Rajiv Gandhi University（拉吉夫·甘地大学）；Tsinghua University（清华大学）；Palmer Research Laboratories（帕勒实验室）；University of Chicago（芝加哥大学）；Stanford University（斯坦福大学）；Purdue University（普渡大学）；University of Southern California（南加州大学）
Domain-Adapted Small Language Models for Reliable Clinical Triage
Department of Computer Science, Virginia Tech, Blacksburg, VA, US（弗吉尼亚理工学院计算机科学系，弗吉尼亚理工学院，布莱克斯堡，VA，美国）；Children’s National Hospital, Washington DC, USA（儿童医学中心，华盛顿特区，美国）
