Radiologists highly desire fully automated AI for radiology report generation (R2G), yet existing approaches fall short in clinical utility. Reinforcement learning (RL) holds potential to address these shortcomings, but its adoption in this task remains underexplored. In this paper, we revisit RL in terms of data efficiency and optimization effectiveness for R2G tasks. First, we explore the impact of data quantity and quality on the performance of RL in medical contexts, revealing that data quality plays a more critical role than quantity. To this end, we propose a diagnostic diversity-based data sampling strategy that enables comparable performance with fewer samples. Second, we observe that the majority of tokens in radiology reports are template-like and diagnostically uninformative, whereas the low frequency of clinically critical tokens heightens the risk of being overlooked during optimization. To tackle this, we introduce Diagnostic Token-weighted Policy Optimization (DiTPO), which directly optimizes for clinical accuracy by using a diagnostic F1 score as the reward signal. Unlike standard RL approaches that treat all tokens equally, DiTPO explicitly models the varying importance of different tokens through rule- or gradient-based mechanisms to prioritize clinically relevant content. Extensive experiments on the MIMIC-CXR, IU-Xray, and CheXpert Plus datasets demonstrate that our framework achieves state-of-the-art (SOTA) performance while requiring substantially fewer training samples in RL. Notably, on MIMIC-CXR, our framework attains an F1 score of 0.516 using only 20% of the RL training samples.
More performant and scalable: Rethinking contrastive vision-language pre-training of radiology in the LLM era
School of Biomedical Engineering, Division of Life Sciences;Medicine, University of Science;Technology of China (USTC), Hefei Anhui, 230026, China Center for Medical Imaging, Robotics, Analytic Computing \& Learning (MIRACLE), Suzhou Institute for Advance Research, USTC, Suzhou Jiangsu, 215123, China The First Affiliated Hospital of USTC, Division of Life Sciences;Medicine, USTC, Hefei Anhui, 230001, China Jiangsu Provincial Key Laboratory of Multimodal Digital Twin Technology, Suzhou Jiangsu, 215123, China State Key Laboratory of Precision
本文提出利用 LLMs 自动从放射学报告中提取诊断标签以低成本构建大规模“银标准”数据集,从而促进大规模监督预训练,仅使用 3D ResNet-18 结合原始 CLIP 训练便在零样本诊断和跨模态检索任务上取得了最先进的性能。
Rethinking Radiology Report Generation via Causal Inspired Counterfactual Augmentation
Nanjing University(南京大学);South China Agricultural University(华南农业大学);Nankai University(南开大学);The First Affiliated Hospital, Sun Yat-sen University(中山大学附属第一医院);Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院)
Multimodal Large Language Model driven Radiology Report Generation with Clinical Knowledge Enhancement
College of Electronic and Information Engineering, Tongji University(同济大学电子与信息工程学院);Shanghai Research Institute for Intelligent Autonomous Systems, Tongji University(同济大学上海自主智能无人系统科学中心)
RadThinking: A Dataset for Longitudinal Clinical Reasoning in Radiology
Department of Computer Science, Johns Hopkins University(约翰霍普金斯大学计算机科学系);Clinic of Radiology and Nuclear Medicine, University Hospital Basel(巴塞尔大学医院放射科与核医学科);Department of Oncology, Johns Hopkins School of Medicine(约翰霍普金斯医学院肿瘤科)
Rethinking Whole-Body CT Image Interpretation: An Abnormality-Centric Approach
School of Artificial Intelligence, Shanghai Jiao Tong University(上海交通大学人工智能学院);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Department of Radiology, Renmin Hospital of Wuhan University(武汉大学仁医院放射科)
RADIANT-PET: Reasoning-Augmented PET/CT Lesion Segmentation with Large Language Models and Reinforcement Learning
Division of Hematology, The Ohio State University Comprehensive Cancer Center(血液科,俄亥俄州立大学综合癌症中心);Department of Radiology, Mahidol University(医学放射科,玛希多大学);Department of Radiology, Mettapracharak Hospital(医学放射科,Mettapracharak医院);Department of Radiation Oncology, The Ohio State University Comprehensive Cancer Center(放射肿瘤科,俄亥俄州立大学综合癌症中心)
Longitudinal Data and a Semantic Similarity Reward for Chest X-Ray Report Generation
The Australian e-Health Research Centre, CSIRO Health and Biosecurity(澳大利亚联邦科学与工业研究组织健康与生物安全部澳大利亚电子健康研究中心);School of Electrical Engineering & Computer Science, University of Queensland(昆士兰大学电气工程与计算机科学学院);Imaging Associates(影像协会);St Vincent’s Hospital(圣文森特医院);Monash Health(莫纳什医疗中心)
UR$^{2}$-MLLM: Uncertainty-aware Revisit Reasoning in Multimodal Large Language Models for Radiology Report Generation
MedVisAI Lab(MedVisAI实验室);Lee Kong Chian School of Medicine, Nanyang Technological University(南洋理工大学李光前医学院);Centre of AI in Medicine, Singapore(新加坡医学人工智能中心);AI Thrust, The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州)AI方向);Ruijin Hospital, Shanghai Jiao Tong University School of Medicine(上海交通大学医学院附属瑞金医院)
Rethinking Patient Education as Multi-turn Multi-modal Interaction
VA Bedford Health Care(VA贝德福德医疗中心);UMass Amherst(马萨诸塞大学阿默斯特分校);UMass Lowell(马萨诸塞大学洛厄尔分校);Yale University(耶鲁大学);National University of Singapore(新加坡国立大学);Yale School of Medicine(耶鲁医学院)
PosterCraft: Rethinking High-Quality Aesthetic Poster Generation in a Unified Framework
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Meituan(美团);Xiamen University(厦门大学);National University of Singapore(新加坡国立大学);The Hong Kong University of Science and Technology(香港科技大学)