arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2601.08192cs.CV

路由、检索、反思、修复:一种自改进的代理框架,用于医学影像中的视觉检测和语言推理

Route, Retrieve, Reflect, Repair: Self-Improving Agentic Framework for Visual Detection and Linguistic Reasoning in Medical Imaging

1 Center for Computational \& Data Sciences (CCDS), Independent University, Bangladesh (IUB)

Md. Faiyaz Abdullah Sayeedi, Rashedur Rahman, Siam Tahsin Bhuiyan, Sefatul Wasi, Ashraful Islam, Saadia Binte Alam, AKM Mahbubur Rahman

AI总结:

R^4框架通过路由、检索、反思和修复四个代理提升医学影像分析的准确性和可靠性。

摘要:

医学影像分析日益依赖于大型视觉-语言模型(VLMs),但大多数系统仍停留在单次传递的黑色盒子里,对推理、安全性和空间定位的控制有限。我们提出了R^4,一种代理框架,将医学影像工作流程分解为四个协调的代理:路由器从图像、患者病史和元数据中配置任务和专业化意识的提示;检索器利用示例记忆和pass@k采样联合生成自由文本报告和边界框;反思器对每个草稿-框对进行批评,以识别关键临床错误模式(否定、侧向性、无支持的主张、矛盾、缺失发现和定位错误);修复器在目标约束下迭代修订叙述和空间输出,同时为未来病例筛选高质量的示例。在胸片分析上实例化,使用多种现代VLM后端,并在报告生成和弱监督检测上评估,R^4在强单VLM基线上的LLM-as-a-Judge分数提高了约+1.7至+2.5分,在mAP50上提高了+2.5至+3.5个绝对点,无需任何基于梯度的微调。这些结果表明,代理路由、反思和修复可以将强大但脆弱的VLMs转化为更可靠且更基础的临床影像解释工具。我们的代码可在:https://github.com/faiyazabdullah/MultimodalMedAgent上找到。

英文摘要:

Medical image analysis increasingly relies on large vision-language models (VLMs), yet most systems remain single-pass black boxes that offer limited control over reasoning, safety, and spatial grounding. We propose R^4, an agentic framework that decomposes medical imaging workflows into four coordinated agents: a Router that configures task- and specialization-aware prompts from the image, patient history, and metadata; a Retriever that uses exemplar memory and pass@k sampling to jointly generate free-text reports and bounding boxes; a Reflector that critiques each draft-box pair for key clinical error modes (negation, laterality, unsupported claims, contradictions, missing findings, and localization errors); and a Repairer that iteratively revises both narrative and spatial outputs under targeted constraints while curating high-quality exemplars for future cases. Instantiated on chest X-ray analysis with multiple modern VLM backbones and evaluated on report generation and weakly supervised detection, R^4 consistently boosts LLM-as-a-Judge scores by roughly +1.7-+2.5 points and mAP50 by +2.5-+3.5 absolute points over strong single-VLM baselines, without any gradient-based fine-tuning. These results show that agentic routing, reflection, and repair can turn strong but brittle VLMs into more reliable and better grounded tools for clinical image interpretation. Our code can be found at: https://github.com/faiyazabdullah/MultimodalMedAgent

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑