基于区域引导的指称与定位大语言模型用于CT报告生成
Large Language Model with Region-guided Referring and Grounding for CT Report Generation
AI总结：
本文提出Reg2RG框架，通过区域引导的指称与定位并结合大语言模型，解决CT报告生成中忽略局部异常的问题，在多个指标上优于现有方法。
摘要：
计算机断层扫描（CT）报告生成对于辅助放射科医生解读CT体数据至关重要，而这一过程可能耗时且费力。现有方法主要仅考虑整个体数据的全局特征，导致其难以聚焦于特定区域并可能遗漏异常。为解决这一问题，我们提出Reg2RG，这是首个用于CT报告生成的区域引导指称与定位框架，通过关注体数据内的解剖区域来提升诊断性能。具体而言，我们利用通用分割模块的掩码来捕获每个指称区域的局部特征。我们提出一种局部特征解耦（LFD）策略，以极小的计算开销保留局部高分辨率细节。随后，局部特征与全局特征相融合，以在连贯上下文中捕获区域间关系。此外，我们提出一种新颖的区域-报告对齐（RRA）训练策略。该策略利用对指称区域的识别来引导区域特定报告的生成，从而增强模型的指称与定位能力，同时提高报告的可解释性。进一步采用大语言模型（LLM）作为语言解码器，从融合的视觉特征生成报告，促进区域级理解。在两个大规模胸部CT报告数据集上的大量实验证明了我们方法的优越性，其在自然语言生成和临床疗效指标上均优于多个最先进方法，同时保持了良好的可解释性。代码可在 https://github.com/zhi-xuan-chen/Reg2RG 获取。
英文摘要：
Computed tomography (CT) report generation is crucial to assist radiologists in interpreting CT volumes, which can be time-consuming and labor-intensive. Existing methods primarily only consider the global features of the entire volume, making it struggle to focus on specific regions and potentially missing abnormalities. To address this issue, we propose Reg2RG, the first region-guided referring and grounding framework for CT report generation, which enhances diagnostic performance by focusing on anatomical regions within the volume. Specifically, we utilize masks from a universal segmentation module to capture local features for each referring region. A local feature decoupling (LFD) strategy is proposed to preserve the local high-resolution details with little computational overhead. Then the local features are integrated with global features to capture inter-regional relationships within a cohesive context. Moreover, we propose a novel region-report alignment (RRA) training strategy. It leverages the recognition of referring regions to guide the generation of region-specific reports, enhancing the model's referring and grounding capabilities while also improving the report's interpretability. A large language model (LLM) is further employed as the language decoder to generate reports from integrated visual features, facilitating region-level comprehension. Extensive experiments on two large-scale chest CT-report datasets demonstrate the superiority of our method, which outperforms several state-of-the-art methods in terms of both natural language generation and clinical efficacy metrics while preserving promising interpretability. The code is available at https://github.com/zhi-xuan-chen/Reg2RG.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
