Most natural language tasks in the radiology domain use language models pre-trained on biomedical corpus. There are few pretrained language models trained specifically for radiology, and fewer still that have been trained in a low data setting and gone on to produce comparable results in fine-tuning tasks. We present RadLing, a continuously pretrained language model using Electra-small (Clark et al., 2020) architecture, trained using over 500K radiology reports, that can compete with state-of-the-art results for fine tuning tasks in radiology domain. Our main contribution in this paper is knowledge-aware masking which is a taxonomic knowledge-assisted pretraining task that dynamically masks tokens to inject knowledge during pretraining. In addition, we also introduce an knowledge base-aided vocabulary extension to adapt the general tokenization vocabulary to radiology domain.
Radiology reports have been widely used for extraction of various clinically significant information about patients' imaging studies. However, limited research has focused on standardizing the entities to a common radiology-specific vocabulary. Further, no study to date has attempted to leverage RadLex for standardization. In this paper, we aim to normalize a diverse set of radiological entities to RadLex terms. We manually construct a normalization corpus by annotating entities from three types of reports. This contains 1706 entity mentions. We propose two deep learning-based NLP methods base…
Radiology reports contain a diverse and rich set of clinical abnormalities documented by radiologists during their interpretation of the images. Comprehensive semantic representations of radiological findings would enable a wide range of secondary use applications to support diagnosis, triage, outcomes prediction, and clinical research. In this paper, we present a new corpus of radiology reports annotated with clinical findings. Our annotation schema captures detailed representations of pathologic findings that are observable on imaging ("lesions") and other types of clinical problems ("medica…
MRScore: Evaluating Radiology Report Generation with LLM-based Reward System
The University of Sydney（悉尼大学）；Guangzhou University of Chinese Medicine（广州中医药大学）；The University of Adelaide（阿德莱德大学）；The University of Wollongong（卧龙岗大学）
MAIRA-Seg: Enhancing Radiology Report Generation with Segmentation-Aware Multimodal Large Language Models
Microsoft Health Futures（微软健康未来）；Microsoft Research India（微软研究院印度分院）；University of Cambridge（剑桥大学）；Cambridge University Hospitals NHS Foundation Trust（剑桥大学医院国民保健信托基金）
Improving Radiology Report Conciseness and Structure via Local Large Language Models
Department of Machine Learning, H. Lee Moffitt Cancer Center & Research Institute（机器学习系，H. Lee Moffitt癌症中心与研究所在读机构）；Deparmtent of Diagnostic Imaging & Interventional Radiology, H. Lee Moffitt Cancer Center & Research Institute（诊断影像与介入放射学系，H. Lee Moffitt癌症中心与研究所在读机构）；Department of Radiology, James A. Haley VA Hospital（放射学系，James A. Haley退伍军人医院）
CCS: Clinical Consensus Selection for Radiology Report Generation
School of Computing Science, University of Glasgow（格拉斯哥大学计算机科学学院）；School of Electrical and Computer Engineering, University of Sydney（悉尼大学电气与计算机工程学院）；Language Technology Lab, University of Cambridge（剑桥大学语言技术实验室）
University Hospital, LMU Munich（慕尼黑大学附属医院）；LMU Munich（慕尼黑大学）；Comprehensive Pneumology Center (CPC-M)（综合肺病中心（CPC-M））；German Center for Lung Research (DZL)（德国肺研究中心（DZL））；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心（MCML））
The release of ChatGPT, a language model capable of generating text that appears human-like and authentic, has gained significant attention beyond the research community. We expect that the convincing performance of ChatGPT incentivizes users to apply it to a variety of downstream tasks, including prompting the model to simplify their own medical reports. To investigate this phenomenon, we conducted an exploratory case study. In a questionnaire, we asked 15 radiologists to assess the quality of radiology reports simplified by ChatGPT. Most radiologists agreed that the simplified reports were f…
RADAR: Enhancing Radiology Report Generation with Supplementary Knowledge Injection
Department of Computing, The Hong Kong Polytechnic University（香港理工大学计算机系）；Research Institute of Trustworthy Autonomous Systems and Department of Computer Science and Engineering, Southern University of Science and Technology（南方科技大学可信自主系统研究院和计算机科学与工程系）；School of Computer Science, University of Nottingham Ningbo China（宁波大学计算机学院）
MGH Radiology Llama: A Llama 3 70B Model for Radiology
School of Computing, University of Georgia（佐治亚大学计算学院）；Department of Radiology, Massachusetts General Hospital and Harvard Medical School（麻省总医院及哈佛医学院放射科）
Microsoft Research Health Futures（微软研究院健康未来部门）；Microsoft Research India（微软研究院印度分部）；Microsoft Azure AI（微软Azure人工智能部门）；Microsoft Health and Life Sciences（微软健康与生命科学部门）；Department of Radiology, Addenbrooke’s Hospital, Cambridge University Hospitals（剑桥大学医院艾登布鲁克医院放射科）
ChatRadio-Valuer: A Chat Large Language Model for Generalizable Radiology Report Generation Based on Multi-institution and Multi-system Data
Northwestern Polytechnical University（西北工业大学）；The Second Xiangya Hospital, Central South University（中南大学湘雅二医院）；Clinical Research Center for Medical Imaging in Hunan Province（湖南省医学影像临床研究中心）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Institute of Medical Research, Northwestern Polytechnical University（西北工业大学医学研究院）；Glasgow College, University of Electronic Science and Technology of China（电子科技大学格拉斯哥学院）；Central South University（中南大学）
Development and Validation of a Large Language Model for Generating Fully-Structured Radiology Reports
Rensselaer Polytechnic Institute（伦斯勒理工学院）；Massachusetts General Hospital, Harvard Medical School（哈佛医学院麻省总医院）；Wake Forest University School of Medicine（维克森林大学医学院）
RadEx: A Framework for Structured Information Extraction from Radiology Reports based on Large Language Models
Institute for Patient-centered Digital Health, Bern University of Applied Sciences, Biel, Switzerland（以患者为中心的数字健康研究所，伯恩应用科学大学，比尔，瑞士）；ID Suisse AG, St. Gallen, Switzerland（ID瑞士股份有限公司，圣加尔，瑞士）
Style-Aware Radiology Report Generation with RadGraph and Few-Shot Prompting
Stanford University（斯坦福大学）；Jawaharlal Institute of Postgraduate Medical Education and Research (JIPMER)（贾瓦哈拉尔研究生医学教育与研究学院）；Hospital Israelita Albert Einstein（以色列阿尔伯特·爱因斯坦医院）；Johns Hopkins University（约翰斯·霍普金斯大学）；Massachusetts General Hospital（马萨诸塞总医院）；Harvard Medical School（哈佛医学院）
Clinically Grounded Agent-based Report Evaluation: An Interpretable Metric for Radiology Report Generation
New York University（纽约大学）；NYU Langone Health（纽约大学兰格医学中心）；Genentech（基因泰克）；NYU Grossman School of Medicine（纽约大学格罗斯曼医学院）；Wenzhou Medical University（温州医科大学）；Macau University of Science and Technology（澳门科学大学）
Neural image-to-text radiology report generation systems offer the potential to improve radiology reporting by reducing the repetitive process of report drafting and identifying possible medical errors. However, existing report generation systems, despite achieving high performances on natural language generation metrics such as CIDEr or BLEU, still suffer from incomplete and inconsistent generations. Here we introduce two new simple rewards to encourage the generation of factually complete and consistent radiology reports: one that encourages the system to generate radiology domain entities c…
University of Zürich and University Hospital of Zürich（苏黎世大学和苏黎世大学医院）；Kyoto University（京都大学）
Inspired by Curriculum Learning, we propose a consecutive (i.e., image-to-text-to-text) generation framework where we divide the problem of radiology report generation into two steps. Contrary to generating the full radiology report from the image at once, the model generates global concepts from the image in the first step and then reforms them into finer and coherent texts using a transformer architecture. We follow the transformer-based sequence-to-sequence paradigm at each step. We improve upon the state-of-the-art on two benchmark datasets.
The IMPRESSIONS section of a radiology report about an imaging study is a summary of the radiologist's reasoning and conclusions, and it also aids the referring physician in confirming or excluding certain diagnoses. A cascade of tasks are required to automatically generate an abstractive summary of the typical information-rich radiology report. These tasks include acquisition of salient content from the report and generation of a concise, easily consumable IMPRESSIONS section. Prior research on radiology report summarization has focused on single-step end-to-end models -- which subsume the ta…
Hospital De Pediatría, Prof. Dr. Juan P. Garrahan（加尔拉汉儿科医院）
This paper presents a new annotated corpus of 513 anonymized radiology reports written in Spanish. Reports were manually annotated with entities, negation and uncertainty terms and relations. The corpus was conceived as an evaluation resource for named entity recognition and relation extraction algorithms, and as input for the use of supervised methods. Biomedical annotated resources are scarce due to confidentiality issues and associated costs. This work provides some guidelines that could help other researchers to undertake similar tasks.
National Center for Biotechnology Information, National Library of Medicine, National Institutes of Health（美国国立卫生研究院国家医学图书馆国家生物技术信息中心）；Department of Radiology and Imaging Sciences, Clinical Center, National Institutes of Health（美国国立卫生研究院临床中心放射与影像科学系）
Negative and uncertain medical findings are frequent in radiology reports, but discriminating them from positive findings remains challenging for information extraction. Here, we propose a new algorithm, NegBio, to detect negative and uncertain findings in radiology reports. Unlike previous rule-based methods, NegBio utilizes patterns on universal dependencies to identify the scope of triggers that are indicative of negation or uncertainty. We evaluated NegBio on four datasets, including two public benchmarking corpora of radiology reports, a new radiology corpus that we annotated for this wor…
Shenzhen Research Institute of Big Data（深圳大数据研究院）；The Chinese University of Hong Kong (Shenzhen)（香港中文大学（深圳））
Medical imaging is frequently used in clinical practice and trials for diagnosis and treatment. Writing imaging reports is time-consuming and can be error-prone for inexperienced radiologists. Therefore, automatically generating radiology reports is highly desired to lighten the workload of radiologists and accordingly promote clinical automation, which is an essential task to apply artificial intelligence to the medical domain. In this paper, we propose to generate radiology reports with memory-driven Transformer, where a relational memory is designed to record key information of the generati…
The Chinese University of Hong Kong (Shenzhen)（香港中文大学（深圳））；Shenzhen Research Institute of Big Data（深圳大数据研究院）
Medical imaging plays a significant role in clinical practice of medical diagnosis, where the text reports of the images are essential in understanding them and facilitating later treatments. By generating the reports automatically, it is beneficial to help lighten the burden of radiologists and significantly promote clinical automation, which already attracts much attention in applying artificial intelligence to medical domain. Previous studies mainly follow the encoder-decoder paradigm and focus on the aspect of text generation, with few studies considering the importance of cross-modal mapp…
University of Edinburgh（爱丁堡大学）；University of Oxford（牛津大学）
NLP has a significant role in advancing healthcare and has been found to be key in extracting structured information from radiology reports. Understanding recent developments in NLP application to radiology is of significance but recent reviews on this are limited. This study systematically assesses recent literature in NLP applied to radiology reports. Our automated literature search yields 4,799 results using automated filtering, metadata enriching steps and citation search combined with manual review. Our analysis is based on 21 variables including radiology characteristics, NLP methodology…
Stanford University（斯坦福大学）；University of Florida（佛罗里达大学）
Neural abstractive summarization models are able to generate summaries which have high overlap with human references. However, existing models are not optimized for factual correctness, a critical metric in real-world applications. In this work, we develop a general framework where we evaluate the factual correctness of a generated summary by fact-checking it automatically against its reference using an information extraction module. We further propose a training strategy which optimizes a neural summarization model with a factual correctness reward via reinforcement learning. We apply the pro…
University of Washington（华盛顿大学）；George Mason University（乔治梅森大学）
Communication of follow-up recommendations when abnormalities are identified on imaging studies is prone to error. In this paper, we present a natural language processing approach based on deep learning to automatically identify clinically important recommendations in radiology reports. Our approach first identifies the recommendation sentences and then extracts reason, test, and time frame of the identified recommendations. To train our extraction models, we created a corpus of 567 radiology reports annotated for recommendation information. Our extraction models achieved 0.92 f-score for reco…
The University of Texas Health Science Center（德克萨斯大学健康科学中心）；National Library of Medicine, National Institutes of Health（美国国立卫生研究院国家医学图书馆）
We define a representation framework for extracting spatial information from radiology reports (Rad-SpRL). We annotated a total of 2000 chest X-ray reports with 4 spatial roles corresponding to the common radiology entities. Our focus is on extracting detailed information of a radiologist's interpretation containing a radiographic finding, its anatomical location, corresponding probable diagnoses, as well as associated hedging terms. For this, we propose a deep learning-based natural language processing (NLP) method involving both word and character-level encodings. Specifically, we utilize a …
Calibrated Confidence Expression for Radiology Report Generation
Computer Aided Medical Procedures, Technical University of Munich, Germany（德国慕尼黑工业大学计算机辅助医疗程序）；Munich Center for Machine Learning (MCML), Germany（德国慕尼黑机器学习中心）；Department of Diagnostic and Interventional Radiology, TUM Klinikum rechts der Isar, Germany（德国慕尼黑工业大学伊萨尔河右岸医院诊断与介入放射科）；Department of Diagnostic and Interventional Neuroradiology, TUM Klinikum rechts der Isar, Germany（德国慕尼黑工业大学伊萨尔河右岸医院诊断与介入神经放射科）；Department of Diagnostic and Interventional Radiology and Nuclear Medicine, University Medical Center Hamburg-Eppendorf, Germany（德国汉堡-埃彭多夫大学医学中心诊断与介入放射学及核医学科）；AI for Image-Guided Diagnosis and Therapy, Technical University of Munich, Germany（德国慕尼黑工业大学人工智能图像引导诊断与治疗）
German CheXpert Chest X-ray Radiology Report Labeler
Munich Institute of Biomedical Engineering, Technical University of Munich（慕尼黑工业大学慕尼黑生物医学工程研究所）；School of Computation, Information and Technology, Technical University of Munich（慕尼黑工业大学计算、信息与技术学院）；University Hospital, LMU Munich（慕尼黑大学附属医院）；Institute of Neuroradiology, University Hospital, LMU Munich（慕尼黑大学附属医院神经放射学研究所）
