MAC-RRG：用于X射线放射学报告生成的迭代式多智能体协作

中文摘要 AI 辅助

尽管基于大语言模型（LLM）和知识图谱增强的放射学报告生成（RRG）方法取得了显著进展，现有技术仍存在固有缺陷。传统的仅使用LLM的模型缺乏结构化的医学先验知识，导致频繁出现医学幻觉和较低的诊断可解释性。当前的知识图谱增强方案采用静态的单轮知识融合，且仅使用单一来源的知识，无法根据生成反馈进行动态知识更新。本文提出了一种新颖的用于X射线放射学报告生成的多智能体协作迭代框架，称为MAC-RRG。受多智能体技术的启发，我们的框架构建了基于任务解耦和协作推理的闭环优化范式。具体而言，该框架首先通过视觉编码器和基础LLM从输入的X射线图像生成初步放射学报告。随后，一个多模态知识图谱（MM-KG）智能体从医学知识图谱中挖掘结构化的疾病关联和解剖学知识，而一个辅助知识智能体从公共医学数据库中提取非结构化的领域知识。双智能体获取的多源知识被融合并嵌入，以引导LLM迭代地优化初始报告。在主流X射线RRG数据集（包括IU X-ray、MIMIC和CheXpert Plus）上进行的大量定量和定性实验充分验证了我们所提出方法的优越性。源代码和预训练模型已在此https URL上发布。

英文摘要

Despite the remarkable progress of LLM-based and knowledge graph-augmented Radiology Report Generation (RRG) methods, existing techniques still suffer from inherent defects. Conventional LLM-only models lack structured medical prior knowledge, resulting in frequent medical hallucinations and low diagnostic interpretability. Current knowledge graph-enhanced schemes adopt static one-round knowledge fusion with single-source knowledge, incapable of dynamic knowledge updating according to generation feedback. This paper proposes a novel Multi-Agent Collaborative iterative framework for X-ray Radiology Report Generation, termed MAC-RRG. Inspired by multi-agent technology, our framework constructs a closed-loop optimization paradigm based on task decoupling and collaborative reasoning. Specifically, the framework first generates a preliminary radiology report from input X-ray images via a vision encoder and a basic LLM. Subsequently, a multimodal knowledge graph (MM-KG) agent mines structured disease correlation and anatomical knowledge from medical knowledge graphs, while an auxiliary knowledge agent extracts unstructured domain knowledge from public medical databases. The multi-source knowledge acquired by dual agents is fused and embedded to guide the LLM in iteratively refining the initial report. Extensive quantitative and qualitative experiments on mainstream X-ray RRG datasets, including IU X-ray, MIMIC, and CheXpert Plus, fully verify the superiority of our proposed method. The source code and pre-trained models have been released on https://github.com/Event-AHU/Medical_Image_Analysis

发表机构

• Anhui University（安徽大学）
• The First Affiliated Hospital of Anhui Medical University（安徽医科大学第一附属医院）
• Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））
• Peng Cheng Laboratory（鹏城实验室）

机构由 AI 辅助整理，请以论文原文为准。