arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2511.11066cs.CVcs.AIcs.CL

S2D-ALIGN：用于解剖学基础放射报告生成的浅到深辅助学习

S2D-ALIGN: Shallow-to-Deep Auxiliary Learning for Anatomically-Grounded Radiology Report Generation

Jiechao Gao, Chang Liu, Yuangang Li

阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对放射报告生成中标准监督微调缺乏解剖学基础对齐的问题，提出S2D-ALIGN从浅到深辅助学习方法，通过多粒度辅助信号和记忆适配器实现解剖锚定对齐，在公共基准上取得最优性能。

摘要：

放射报告生成（RRG）旨在从放射图像自动生成诊断报告。现有方法利用多模态大型语言模型（MLLMs）强大的跨模态生成能力，主要通过监督微调（SFT）优化放射图像与报告的跨模态对齐。然而，标准SFT范式仅对图像-文本对进行实例级对齐，无法建立解剖学基础的对齐，报告的模板化特性常导致生成质量欠佳。为此，我们提出S2D-ALIGN：一种从浅到深的辅助学习方法，通过不同粒度的辅助信号建立解剖学基础对齐。该方法采用浅到深策略，逐步丰富对齐过程：先进行粗略的放射图像-报告配对，再引入参考报告提供实例级指导，最终利用关键短语将生成锚定在特定解剖细节上。为连接不同对齐阶段，我们引入基于记忆的适配器实现特征共享，整合粗粒度和细粒度指导。在公共基准MIMIC-CXR和IU X-Ray上的实验表明，S2D-ALIGN相较于现有方法达到了最先进性能。消融研究验证了多阶段、辅助引导方法的有效性，为增强复杂多模态生成任务中的锚定能力提供了可行方向。

英文摘要：

Radiology Report Generation (RRG) aims to automatically generate diagnostic reports from radiology images. To achieve this, existing methods have leveraged the powerful cross-modal generation capabilities of Multimodal Large Language Models (MLLMs), primarily focusing on optimizing cross-modal alignment between radiographs and reports through Supervised Fine-Tuning (SFT). However, by only performing instance-level alignment with the image-text pairs, the standard SFT paradigm fails to establish anatomically-grounded alignment, where the templated nature of reports often leads to sub-optimal generation quality. To address this, we propose \textsc{S2D-Align}, a novel SFT paradigm that establishes anatomically-grounded alignment by leveraging auxiliary signals of varying granularities. \textsc{S2D-Align} implements a shallow-to-deep strategy, progressively enriching the alignment process: it begins with the coarse radiograph-report pairing, then introduces reference reports for instance-level guidance, and ultimately utilizes key phrases to ground the generation in specific anatomical details. To bridge the different alignment stages, we introduce a memory-based adapter that empowers feature sharing, thereby integrating coarse and fine-grained guidance. For evaluation, we conduct experiments on the public \textsc{MIMIC-CXR} and \textsc{IU X-Ray} benchmarks, where \textsc{S2D-Align} achieves state-of-the-art performance compared to existing methods. Ablation studies validate the effectiveness of our multi-stage, auxiliary-guided approach, highlighting a promising direction for enhancing grounding capabilities in complex, multi-modal generation tasks.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑