arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2601.17151cs.CVcs.CL
收藏

通过多模态强化学习扩展医学影像报告生成

Scaling medical imaging report generation with multimodal reinforcement learning

Microsoft Research(微软研究院)

Qianchu Liu, Sheng Zhang, Guanghui Qin, Yu Gu, Ying Jin, Sam Preston, Yanbo Xu, Sid Kiblawi, Wen-wai Yim, Timothy Ossowski, Tristan Naumann, Mu Wei, Hoifung Poon

AI总结:

本文提出UniRG框架,通过多模态强化学习提升医学影像报告生成的性能和泛化能力,在CXR报告生成中取得新的SOTA成绩。

摘要:

前沿模型在理解和推理自然语言文本方面展示了显著的能力,但在多模态理解和推理,尤其是在生物医学等高价值垂直领域方面仍存在重大能力差距。医学影像报告生成是典型的例子。有监督微调可以显著提高性能,但容易过度拟合表面的模板模式。在本文中,我们引入了通用报告生成(UniRG)作为医学影像报告生成的一般框架。通过利用强化学习作为统一机制,直接优化为端应用设计的评估指标,UniRG可以显著优于有监督微调,并在不同机构和临床实践之间实现持续的泛化。我们训练了UniRG-CXR在公开可用的胸部X光(CXR)数据上,并在CXR报告生成中进行了彻底的评估,使用严格评估场景。在权威的ReXrank基准上,UniRG-CXR设定了新的整体SOTA,远远超越了先前的最先进水平。

英文摘要:

Frontier models have demonstrated remarkable capabilities in understanding and reasoning with natural-language text, but they still exhibit major competency gaps in multimodal understanding and reasoning especially in high-value verticals such as biomedicine. Medical imaging report generation is a prominent example. Supervised fine-tuning can substantially improve performance, but they are prone to overfitting to superficial boilerplate patterns. In this paper, we introduce Universal Report Generation (UniRG) as a general framework for medical imaging report generation. By leveraging reinforcement learning as a unifying mechanism to directly optimize for evaluation metrics designed for end applications, UniRG can significantly improve upon supervised fine-tuning and attain durable generalization across diverse institutions and clinical practices. We trained UniRG-CXR on publicly available chest X-ray (CXR) data and conducted a thorough evaluation in CXR report generation with rigorous evaluation scenarios. On the authoritative ReXrank benchmark, UniRG-CXR sets new overall SOTA, outperforming prior state of the art by a wide margin. We release our model at https://huggingface.co/microsoft/UniRG-CXR.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑