UR²-MLLM：面向放射科报告生成的多模态大语言模型中基于不确定性感知的重访推理
UR$^{2}$-MLLM: Uncertainty-aware Revisit Reasoning in Multimodal Large Language Models for Radiology Report Generation
浏览论文内容
中文总结 AI 辅助
该研究针对放射科报告生成任务，提出UR²-MLLM框架，通过动态重访不确定区域的机制实现最优性能，提升报告的可靠性与临床适配性。
中文摘要 AI 辅助
放射科医生通过迭代且选择性地重访可疑区域来完善诊断解读，从而生成诊断报告。近期用于放射科报告生成（RRG）的多模态大语言模型（MLLM）已从纯文本推理转向“基于图像思考”的范式，将视觉证据融入推理过程。然而，现有方法提供静态视觉证据，推理过程中缺乏动态重访机制，忽略了放射科医生如何重新检查不确定性观察结果。为此，我们提出一种不确定性感知重访推理多模态大语言模型（UR²-MLLM）框架，用于RRG时在推理过程中动态重访不确定区域。UR²-MLLM首先通过在不确定性感知数据集上训练具备不确定性感知能力；随后结合检测-复制机制构建多模态推理轨迹数据集，以指导重访的时机与位置；最后通过视觉 grounding 奖励强化学习优化该行为，使重访区域与相应解剖结构对齐。在MIMIC-CXR和IU-Xray上的实验表明，UR²-MLLM实现了最优性能，凸显了不确定性感知视觉重访推理对生成可靠且符合临床要求报告的价值。
英文摘要
Radiologists generate diagnostic reports through iterative and selective revisiting of suspicious regions to refine their interpretations. Recent multimodal large language models (MLLMs) for radiology report generation (RRG) have shifted from text-only reasoning toward a ``Thinking-with-Images'' paradigm, incorporating visual evidence into the reasoning process. However, existing methods provide static visual evidence without a dynamic revisit mechanism during reasoning, neglecting how radiologists re-examine uncertain observations. To this end, we propose an Uncertainty-aware Revisit Reasoning MLLM (UR$^{2}$-MLLM) framework that dynamically revisits uncertain regions during reasoning for RRG. UR$^{2}$-MLLM is first equipped with uncertainty perception by training on an uncertainty-aware dataset. We then construct a multimodal reasoning trajectory dataset together with a detect-and-copy mechanism, which guides when and where to revisit. Finally, a visual grounding reward refines this behavior through reinforcement learning, aligning the revisited regions with corresponding anatomical structures. Experiments on MIMIC-CXR and IU-Xray show that UR$^{2}$-MLLM achieves state-of-the-art performance, highlighting the value of uncertainty-aware visual revisit reasoning for reliable and clinically aligned report generation.
发表机构
- MedVisAI Lab（MedVisAI实验室）
- Lee Kong Chian School of Medicine, Nanyang Technological University（南洋理工大学李光前医学院）
- Centre of AI in Medicine, Singapore（新加坡医学人工智能中心）
- AI Thrust, The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州）AI方向）
- Ruijin Hospital, Shanghai Jiao Tong University School of Medicine（上海交通大学医学院附属瑞金医院）
机构由 AI 辅助整理，请以论文原文为准。
