Paying Per-label Attention for Multi-label Extraction from Radiology Reports
Canon Medical Research Europe（佳能医疗欧洲研究中心）；University of St Andrews（圣安德鲁斯大学）；Institute of Neuroscience & Psychology, University of Glasgow（格拉斯哥大学神经科学与心理学研究所）；University of Edinburgh（爱丁堡大学）
Patrick Schrempf, Hannah Watson, Shadia Mikhael, Maciej Pajak, Matúš Falis, Aneta Lisowska, Keith W. Muir, David Harris-Birtill, Alison Q. O'Neil
Training medical image analysis models requires large amounts of expertly annotated data which is time-consuming and expensive to obtain. Images are often accompanied by free-text radiology reports which are a rich source of information. In this paper, we tackle the automated extraction of structured labels from head CT reports for imaging of suspected stroke patients, using deep learning. Firstly, we propose a set of 31 labels which correspond to radiographic findings (e.g. hyperdensity) and clinical impressions (e.g. haemorrhage) related to neurological abnormalities. Secondly, inspired by previous work, we extend existing state-of-the-art neural network models with a label-dependent attention mechanism. Using this mechanism and simple synthetic data augmentation, we are able to robustly extract many labels with a single model, classified according to the radiologist's reporting (positive, uncertain, negative). This approach can be used in further research to effectively extract many labels from medical text.
AI-Driven Radiology Report Generation for Traumatic Brain Injuries
Higher School of Digital Economics, Manouba, University of Manouba, Tunisia（数字经济学高等学院，曼布巴，曼布巴大学，突尼斯）；Higher Institute of Management of Tunis, University of Tunis, Tunisia（突尼斯管理高等学院，突尼斯大学，突尼斯）；Emergency Department Charles Nicolle Hospital, Tunis El Manar University, Tunisia（查尔斯·尼科勒医院急诊部，突尼斯埃尔曼纳大学，突尼斯）
University of Washington（华盛顿大学）；George Mason University（乔治梅森大学）
Communication of follow-up recommendations when abnormalities are identified on imaging studies is prone to error. In this paper, we present a natural language processing approach based on deep learning to automatically identify clinically important recommendations in radiology reports. Our approach first identifies the recommendation sentences and then extracts reason, test, and time frame of the identified recommendations. To train our extraction models, we created a corpus of 567 radiology reports annotated for recommendation information. Our extraction models achieved 0.92 f-score for reco…
Extraction of relevant pathological terms from radiology reports is important for correct image label generation and disease population studies. In this letter, we compare the performance of some known application program interface (APIs) for the task of thoracic abnormality extraction from radiology reports. We explored several medical domain specific annotation tools like Medical Text Indexer(MTI) with Non-MEDLINE and Mesh On Demand(MOD) options and generic Natural Language Understanding (NLU) API provided by the IBM cloud. Our results show that although MTI and MOD are intended for extracti…
Radiology reports contain a diverse and rich set of clinical abnormalities documented by radiologists during their interpretation of the images. Comprehensive semantic representations of radiological findings would enable a wide range of secondary use applications to support diagnosis, triage, outcomes prediction, and clinical research. In this paper, we present a new corpus of radiology reports annotated with clinical findings. Our annotation schema captures detailed representations of pathologic findings that are observable on imaging ("lesions") and other types of clinical problems ("medica…
The University of Texas Health Science Center（德克萨斯大学健康科学中心）；National Library of Medicine, National Institutes of Health（美国国立卫生研究院国家医学图书馆）
We define a representation framework for extracting spatial information from radiology reports (Rad-SpRL). We annotated a total of 2000 chest X-ray reports with 4 spatial roles corresponding to the common radiology entities. Our focus is on extracting detailed information of a radiologist's interpretation containing a radiographic finding, its anatomical location, corresponding probable diagnoses, as well as associated hedging terms. For this, we propose a deep learning-based natural language processing (NLP) method involving both word and character-level encodings. Specifically, we utilize a …
Pretrained Transformer based models finetuned on domain specific corpora have changed the landscape of NLP. Generally, if one has multiple tasks on a given dataset, one may finetune different models or use task specific adapters. In this work, we show that a multi-task model can beat or achieve the performance of multiple BERT-based models finetuned on various tasks and various task specific adapter augmented BERT-based models. We validate our method on our internal radiologist's report dataset on cervical spine. We hypothesize that the tasks are semantically close and related and thus multita…
RadEx: A Framework for Structured Information Extraction from Radiology Reports based on Large Language Models
Institute for Patient-centered Digital Health, Bern University of Applied Sciences, Biel, Switzerland（以患者为中心的数字健康研究所，伯恩应用科学大学，比尔，瑞士）；ID Suisse AG, St. Gallen, Switzerland（ID瑞士股份有限公司，圣加尔，瑞士）
German CheXpert Chest X-ray Radiology Report Labeler
Munich Institute of Biomedical Engineering, Technical University of Munich（慕尼黑工业大学慕尼黑生物医学工程研究所）；School of Computation, Information and Technology, Technical University of Munich（慕尼黑工业大学计算、信息与技术学院）；University Hospital, LMU Munich（慕尼黑大学附属医院）；Institute of Neuroradiology, University Hospital, LMU Munich（慕尼黑大学附属医院神经放射学研究所）
School of Biomedical Engineering, Capital Medical University（首都医科大学生物医学工程学院）；Beijing Friendship Hospital, Capital Medical University（首都医科大学附属北京友谊医院）；Tsinghua University（清华大学）
Despite the rapid development of natural language processing (NLP) implementation in electronic medical records (EMRs), Chinese EMRs processing remains challenging due to the limited corpus and specific grammatical characteristics, especially for radiology reports. In this study, we designed an NLP pipeline for the direct extraction of clinically relevant features from Chinese radiology reports, which is the first key step in computer-aided radiologic diagnosis. The pipeline was comprised of named entity recognition, synonyms normalization, and relationship extraction to finally derive the rad…
Extracting structured clinical information from free-text radiology reports can enable the use of radiology report information for a variety of critical healthcare applications. In our work, we present RadGraph, a dataset of entities and relations in full-text chest X-ray radiology reports based on a novel information extraction schema we designed to structure radiology reports. We release a development dataset, which contains board-certified radiologist annotations for 500 radiology reports from the MIMIC-CXR dataset (14,579 entities and 10,889 relations), and a test dataset, which contains t…
Leveraging Prompt-Learning for Structured Information Extraction from Crohn's Disease Radiology Reports in a Low-Resource Language
Faculty of Data and Decision Sciences, Technion - Israel Institute of Technology（以色列理工学院数据与决策科学学院）；Shaare Zedek Medical Center（沙雷泽德克医疗中心）；Faculty of Biomedical Engineering, Technion - Israel Institute of Technology（以色列理工学院生物医学工程学院）；Hospital for Sick Children（病童医院）
提出SMP-BERT提示学习方法，利用报告结构从希伯来语克罗恩病放射学报告中提取结构化信息，显著优于传统微调，尤其在罕见病症检测上（AUC 0.99 vs 0.94，F1 0.84 vs 0.34），推动低资源语言的AI诊断。
Hospital De Pediatría, Prof. Dr. Juan P. Garrahan（加尔拉汉儿科医院）
This paper presents a new annotated corpus of 513 anonymized radiology reports written in Spanish. Reports were manually annotated with entities, negation and uncertainty terms and relations. The corpus was conceived as an evaluation resource for named entity recognition and relation extraction algorithms, and as input for the use of supervised methods. Biomedical annotated resources are scarce due to confidentiality issues and associated costs. This work provides some guidelines that could help other researchers to undertake similar tasks.
Shiraz University（设拉子大学）；Shiraz University of Medical Sciences（设拉子医科大学）
Since radiology reports needed for clinical practice and research are written and stored in free-text narrations, extraction of relative information for further analysis is difficult. In these circumstances, natural language processing (NLP) techniques can facilitate automatic information extraction and transformation of free-text formats to structured data. In recent years, deep learning (DL)-based models have been adapted for NLP experiments with promising results. Despite the significant potential of DL models based on artificial neural networks (ANN) and convolutional neural networks (CNN)…
School of Biomedical Informatics（生物医学信息学院）；McGovern Medical School（麦戈文医学院）；The University of Texas Health Science Center at Houston（得克萨斯大学休斯顿健康科学中心）
Classifying fine-grained ischemic stroke phenotypes relies on identifying important clinical information. Radiology reports provide relevant information with context to determine such phenotype information. We focus on stroke phenotypes with location-specific information: brain region affected, laterality, stroke stage, and lacunarity. We use an existing fine-grained spatial information extraction system--Rad-SpatialNet--to identify clinically important information and apply simple domain rules on the extracted information to classify phenotypes. The performance of our proposed approach is pro…
Acquiring high-quality annotations in medical imaging is usually a costly process. Automatic label extraction with natural language processing (NLP) has emerged as a promising workaround to bypass the need of expert annotation. Despite the convenience, the limitation of such an approximation has not been carefully examined and is not well understood. With a challenging set of 1,000 chest X-ray studies and their corresponding radiology reports, we show that there exists a surprisingly large discrepancy between what radiologists visually perceive and what they clinically report. Furthermore, wit…
Indian Institute of Technology Bombay（印度理工学院孟买分校）；Augnito India Pvt Ltd（印度Augnito私人有限公司）
Conventionally, the radiologist prepares the diagnosis notes and shares them with the transcriptionist. Then the transcriptionist prepares a preliminary formatted report referring to the notes, and finally, the radiologist reviews the report, corrects the errors, and signs off. This workflow causes significant delays and errors in the report. In current research work, we focus on applications of NLP techniques like Information Extraction (IE) and domain-specific Knowledge Graph (KG) to automatically generate radiology reports from radiologist's dictation. This paper focuses on KG construction …
Hierarchical Section Matching Prediction (HSMP) BERT for Fine-Grained Extraction of Structured Data from Hebrew Free-Text Radiology Reports in Crohn's Disease
Faculty of Data and Decision Sciences, Technion – Israel Institute of Technology（1 数据与决策科学学院，技术离子-以色列理工学院）；Faculty of Biomedical Engineering, Technion – Israel Institute of Technology（2 生物医学工程学院，技术离子-以色列理工学院）；Juliet Keidan Institute of Pediatric Gastroenterology, Hepatology and Nutrition, The Eisenberg R&D Authority, Shaare Zedek Medical Center, The Hebrew University School of Medicine（3 肝胆营养研究所，Eisenberg R&D机构，Shaare Zedek医疗中心，希伯来大学医学院）；Pediatric Radiology Unit, Radiology Department, The Eisenberg R&D Authority, Shaare Zedek Medical Center, The Hebrew University of Jerusalem（4 儿童放射科单位，放射科部门，Eisenberg R&D机构，Shaare Zedek医疗中心，希伯来大学杰里科分校）
The extraction of labels from radiology text reports enables large-scale training of medical imaging models. Existing approaches to report labeling typically rely either on sophisticated feature engineering based on medical domain knowledge or manual annotations by experts. In this work, we introduce a BERT-based approach to medical image report labeling that exploits both the scale of available rule-based systems and the quality of expert annotations. We demonstrate superior performance of a biomedically pretrained BERT model first trained on annotations of a rule-based labeler and then finet…
Radiology reports have been widely used for extraction of various clinically significant information about patients' imaging studies. However, limited research has focused on standardizing the entities to a common radiology-specific vocabulary. Further, no study to date has attempted to leverage RadLex for standardization. In this paper, we aim to normalize a diverse set of radiological entities to RadLex terms. We manually construct a normalization corpus by annotating entities from three types of reports. This contains 1706 entity mentions. We propose two deep learning-based NLP methods base…
Neural image-to-text radiology report generation systems offer the potential to improve radiology reporting by reducing the repetitive process of report drafting and identifying possible medical errors. However, existing report generation systems, despite achieving high performances on natural language generation metrics such as CIDEr or BLEU, still suffer from incomplete and inconsistent generations. Here we introduce two new simple rewards to encourage the generation of factually complete and consistent radiology reports: one that encourages the system to generate radiology domain entities c…
National Center for Biotechnology Information, National Library of Medicine, National Institutes of Health（美国国立卫生研究院国家医学图书馆国家生物技术信息中心）；Department of Radiology and Imaging Sciences, Clinical Center, National Institutes of Health（美国国立卫生研究院临床中心放射与影像科学系）
Negative and uncertain medical findings are frequent in radiology reports, but discriminating them from positive findings remains challenging for information extraction. Here, we propose a new algorithm, NegBio, to detect negative and uncertain findings in radiology reports. Unlike previous rule-based methods, NegBio utilizes patterns on universal dependencies to identify the scope of triggers that are indicative of negation or uncertainty. We evaluated NegBio on four datasets, including two public benchmarking corpora of radiology reports, a new radiology corpus that we annotated for this wor…
CodaMetrix（科达矩阵）；Amazon（亚马逊）；Mass General Physicians Organization（麻省总医院医师组织）
Automatic structuring of electronic medical records is of high demand for clinical workflow solutions to facilitate extraction, storage, and querying of patient care information. However, developing a scalable solution is extremely challenging, specifically for radiology reports, as most healthcare institutes use either no template or department/institute specific templates. Moreover, radiologists' reporting style varies from one to another as sentences are telegraphic and do not follow general English grammar rules. We present an ensemble method that consolidates the predictions of three mode…
Stanford University（斯坦福大学）；University of Florida（佛罗里达大学）
Neural abstractive summarization models are able to generate summaries which have high overlap with human references. However, existing models are not optimized for factual correctness, a critical metric in real-world applications. In this work, we develop a general framework where we evaluate the factual correctness of a generated summary by fact-checking it automatically against its reference using an information extraction module. We further propose a training strategy which optimizes a neural summarization model with a factual correctness reward via reinforcement learning. We apply the pro…
The IMPRESSIONS section of a radiology report about an imaging study is a summary of the radiologist's reasoning and conclusions, and it also aids the referring physician in confirming or excluding certain diagnoses. A cascade of tasks are required to automatically generate an abstractive summary of the typical information-rich radiology report. These tasks include acquisition of salient content from the report and generation of a concise, easily consumable IMPRESSIONS section. Prior research on radiology report summarization has focused on single-step end-to-end models -- which subsume the ta…
Weill Cornell Medicine（威尔康奈尔医学院）；UAMS College of Medicine（阿肯色大学医学科学分校医学院）
Utilizing clinical texts in survival analysis is difficult because they are largely unstructured. Current automatic extraction models fail to capture textual information comprehensively since their labels are limited in scope. Furthermore, they typically require a large amount of data and high-quality expert annotations for training. In this work, we present a novel method of using BERT-based hidden layer representations of clinical texts as covariates for proportional hazards models to predict patient survival outcomes. We show that hidden layers yield notably more accurate predictions than p…
Cupertino High School（库比蒂诺高中）；Trillium Health Partners（延龄草健康伙伴）；Enlitic Inc.（Enlitic公司）
Obtaining datasets labeled to facilitate model development is a challenge for most machine learning tasks. The difficulty is heightened for medical imaging, where data itself is limited in accessibility and labeling requires costly time and effort by trained medical specialists. Medical imaging studies, however, are often accompanied by a medical report produced by a radiologist, identifying important features on the corresponding scan for other physicians not specifically trained in radiology. We propose a methodology for approximating image-level labels for radiology studies from associated …
Agent-Based Uncertainty Awareness Improves Automated Radiology Report Labeling with an Open-Source Large Language Model
Faculty of Biomedical Engineering, Technion - Israel Institute of Technology（以色列理工学院 生物医学工程学院）；Faculty of Data and Decision Sciences, Technion - Israel Institute of Technology（以色列理工学院 数据与决策科学学院）；Juliet Keidan Institute of Pediatric Gastroenterology Hepatology and Nutrition, Shaare Zedek Medical Center, The Hebrew University School of Medicine（沙尔泽德克医疗中心 朱丽叶·凯丹儿科胃肠病学、肝病学与营养学研究所；希伯来大学医学院）；The Eisenberg R&D Authority, Shaare Zedek Medical Center, The Hebrew University of Jerusalem（沙尔泽德克医疗中心 艾森伯格研发管理局；耶路撒冷希伯来大学）
RadAnnotate: Large Language Models for Efficient and Reliable Radiology Report Annotation
Department of Computer Science, University of California, Davis, CA, USA（加州大学戴维斯分校计算机科学系）；Department of Radiology, University of California, Davis, CA, USA（加州大学戴维斯分校放射学系）
University of Latvia（拉脱维亚大学）；Charles University（查理大学）
Processing of multi-word expressions (MWEs) is a known problem for any natural language processing task. Even neural machine translation (NMT) struggles to overcome it. This paper presents results of experiments on investigating NMT attention allocation to the MWEs and improving automated translation of sentences that contain MWEs in English->Latvian and English->Czech NMT systems. Two improvement strategies were explored -(1) bilingual pairs of automatically extracted MWE candidates were added to the parallel corpus used to train the NMT system, and (2) full sentences containing the automatic…
ReportMedSAM: Guiding Segmentation Through Radiology Reports
School of Engineering, College of Engineering and Physical Sciences, University of Birmingham（工程学院，工程与物理科学学院，伯明翰大学）；Department of Cancer and Genomic Sciences, College of Medicine and Health, University of Birmingham（癌症与基因组科学系，医学与健康学院，伯明翰大学）；Department of Computer Science, Faculty of Science and Engineering, University of Manchester（计算机科学系，科学与工程学院，曼彻斯特大学）；The Division of Informatics, Imaging and Data Science, Faculty of Biology, Medicine, and Health, University of Manchester（信息学、成像与数据科学部，生物、医学与健康学院，曼彻斯特大学）
Automated Labeling of German Chest X-Ray Radiology Reports using Deep Learning
Munich Institute of Biomedical Engineering, Technical University of Munich（慕尼黑工业大学慕尼黑生物医学工程研究所）；School of Computation, Information and Technology, Technical University of Munich（慕尼黑工业大学计算、信息与技术学院）；University Hospital, LMU Munich（慕尼黑大学附属医院）；Institute of Neuroradiology, University Hospital, LMU Munich（慕尼黑大学附属医院神经放射学研究所）
University of Washington（华盛顿大学）；George Mason University（乔治梅森大学）
Medical imaging is critical to the diagnosis and treatment of numerous medical problems, including many forms of cancer. Medical imaging reports distill the findings and observations of radiologists, creating an unstructured textual representation of unstructured medical images. Large-scale use of this text-encoded information requires converting the unstructured text to a structured, semantic representation. We explore the extraction and normalization of anatomical information in radiology reports that is associated with radiological findings. We investigate this extraction and normalization …
Intelligent Documentation in Medical Education: Can AI Replace Manual Case Logging?
Department of Computer Science, University of California, Davis, CA, USA（计算机科学系，加州大学戴维斯分校）；Department of Radiology, University of California, Davis, CA, USA（放射学系，加州大学戴维斯分校）
The University of Texas at Austin（德克萨斯大学奥斯汀分校）；University of Pennsylvania（宾夕法尼亚大学）；Galileo CDS Inc.（Galileo CDS公司）
Document-level models for information extraction tasks like slot-filling are flexible: they can be applied to settings where information is not necessarily localized in a single sentence. For example, key features of a diagnosis in a radiology report may not be explicitly stated in one place, but nevertheless can be inferred from parts of the report's text. However, these models can easily learn spurious correlations between labels and irrelevant information. This work studies how to ensure that these models make correct inferences from complex text and make those inferences in an auditable wa…
Extracting and Encoding: Leveraging Large Language Models and Medical Knowledge to Enhance Radiological Text Representation
Pontificia Universidad Católica de Chile（智利天主教大学）；University of Pennsylvania（宾夕法尼亚大学）；KU Leuven（鲁汶大学）；Millennium Institute for Intelligent Healthcare Engineering (iHEALTH)（千禧智能医疗工程研究院（iHEALTH））；National Center for Artificial Intelligence (CENIA)（国家人工智能中心（CENIA））
Structural Entities Extraction and Patient Indications Incorporation for Chest X-ray Report Generation
School of Computer Science and Technology, Xidian University（西安电子科技大学计算机科学与技术学院）；Xi'an Key Laboratory of Big Data and Intelligent Vision（西安大数据与智能视觉重点实验室）；Key Laboratory of Collaborative Intelligence Systems, Ministry of Education, Xidian University（教育部协同智能系统重点实验室）；Warren Alpert Medical School, Brown University（布朗大学沃伦·阿尔珀特医学院）；School of Electronic Engineering, Xidian University（西安电子科技大学电子工程学院）；Department of Radiology and Radiological Sciences, Johns Hopkins University School of Medicine（约翰霍普金斯大学医学院放射科）
Duke University School of Medicine（杜克大学医学院）；Duke University（杜克大学）；University of Girona（赫罗纳大学）；South China University of Technology（华南理工大学）；University of Arizona College of Medicine（亚利桑那大学医学院）
Purpose: To develop high throughput multi-label annotators for body (chest, abdomen, and pelvis) Computed Tomography (CT) reports that can be applied across a variety of abnormalities, organs, and disease states. Approach: We used a dictionary approach to develop rule-based algorithms (RBA) for extraction of disease labels from radiology text reports. We targeted three organ systems (lungs/pleura, liver/gallbladder, kidneys/ureters) with four diseases per system based on their prevalence in our dataset. To expand the algorithms beyond pre-defined keywords, attention-guided recurrent neural net…
