arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.12843cs.CV
收藏

MMRad-22K:用于胸部X光报告生成的结构化多模态证据数据集

MMRad-22K: A Structured Multimodal Evidence Dataset for Chest X-ray Report Generation

MoE Key Lab of Artificial Intelligence, AI Institute, School of Computer Science, Shanghai Jiao Tong University(人工智能MOE实验室、人工智能研究院、计算机科学学院、上海交通大学);School of Biomedical Engineering, Division of Life Sciences and Medicine, University of Science and Technology of China (USTC)(生物医学工程学院、生命科学与医学系、中国科学技术大学);Center for Medical Imaging, Robotics, Analytic Computing & Learning (MIRACLE), Suzhou Institute for Advanced Research, USTC(医学影像、机器人、分析计算与学习中心(MIRACLE)、苏州市先进研究院、中国科学技术大学);Department of Radiology, The First Affiliated Hospital, Zhejiang University School of Medicine(放射科、浙江大学医学院第一附属医院)

Yichen Zhao, Zelin Peng, Fenghe Tang, Piao Yang, Yu Huang, Wei Shen

AI总结:

针对胸部X光报告生成中现有资源监督信号碎片化的问题,提出结构化多模态证据数据集MMRad-22K,并基于统一LVLM骨干进行适配,证明结构化多模态证据优于纯文本或边界框证据,在语言和临床指标上表现更优。

摘要:

胸部X光(CXR)报告遵循基于区域的临床工作流程,放射科医生检查解剖区域并将局部发现整合到最终报告中。然而,现有的CXR报告生成资源以碎片化形式提供这些监督信号。我们引入MMRad-22K,一个将区域文本观察、解剖定位坐标、局部图像证据和报告目标组织成结构化多模态证据单元的数据集,用于CXR报告生成。为了推动这一构想,我们首先比较了不同证据格式对报告生成的影响,发现结构化多模态证据通常比纯文本或基于边界框的证据更有用。然后,我们使用MMRad-22K适配统一的LVLM骨干,并证明多模态证据适配在语言和临床导向指标上均优于文本证据适配和端到端适配。在相同的评估协议下,适配模型也达到了与几个开源LVLM参考相当的性能水平。这些结果共同支持MMRad-22K作为实用的结构化多模态资源,用于训练和评估与临床阅读工作流程一致的CXR报告生成。

英文摘要:

Chest X-ray (CXR) reporting follows a region-based clinical workflow in which radiologists inspect anatomical regions and integrate localized findings into a final report. However, existing resources for CXR report generation provide these supervision signals in fragmented forms. We introduce MMRad-22K, a dataset that organizes regional textual observations, anatomical grounding coordinates, localized image evidence, and report targets into structured multimodal evidence units for CXR report generation. To motivate this formulation, we first compare different evidence formats for report generation and find that structured multimodal evidence is generally more useful than text-only or bounding box-based evidence. We then adapt a unified LVLM backbone using MMRad-22K and show that adaptation with multimodal evidence outperforms both textual-evidence adaptation and end-to-end adaptation on language and clinically oriented metrics. Under the same evaluation protocol, the adapted model also reaches a performance level comparable to several open-source LVLM references. Together, these results support MMRad-22K as a practical structured multimodal resource for training and evaluating CXR report generation aligned with clinical reading workflows.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑