病理代理RAG：通过强化学习实现多模态代理检索增强生成用于病理学视觉语言模型
Patho-AgenticRAG: Towards Multimodal Agentic Retrieval-Augmented Generation for Pathology VLMs via Reinforcement Learning
AI总结：
本文提出Patho-AgenticRAG，通过强化学习实现多模态代理检索增强生成，解决病理学视觉语言模型在高分辨率、复杂组织结构和临床语义上的挑战，提升诊断准确性。
摘要：
尽管视觉语言模型（VLMs）在医学影像中表现出强大的泛化能力，但病理学由于超高分辨率、复杂组织结构和微妙的临床语义而面临独特挑战。这些因素使病理学VLMs容易产生幻觉，即生成与视觉证据不一致的输出，从而损害临床信任。现有该领域的RAG方法主要依赖文本知识库，限制了利用诊断视觉线索的能力。为此，我们提出Patho-AgenticRAG，一种基于权威病理学教科书页面嵌入构建的多模态RAG框架。与传统纯文本检索系统不同，它支持联合文本-图像搜索，能够直接检索包含查询文本和相关视觉线索的教科书页面，从而避免关键图像信息的丢失。Patho-AgenticRAG还支持推理、任务分解和多轮搜索交互，提高复杂诊断场景的准确性。实验表明，Patho-AgenticRAG在多重选择诊断和视觉问答等复杂病理任务上显著优于现有多模态模型。我们的项目可在Patho-AgenticRAG仓库中获得：https://github.com/Wenchuan-Zhang/Patho-AgenticRAG。
英文摘要：
Although Vision Language Models (VLMs) have shown strong generalization in medical imaging, pathology presents unique challenges due to ultra-high resolution, complex tissue structures, and nuanced clinical semantics. These factors make pathology VLMs prone to hallucinations, i.e., generating outputs inconsistent with visual evidence, which undermines clinical trust. Existing RAG approaches in this domain largely depend on text-based knowledge bases, limiting their ability to leverage diagnostic visual cues. To address this, we propose Patho-AgenticRAG, a multimodal RAG framework with a database built on page-level embeddings from authoritative pathology textbooks. Unlike traditional text-only retrieval systems, it supports joint text-image search, enabling direct retrieval of textbook pages that contain both the queried text and relevant visual cues, thus avoiding the loss of critical image-based information. Patho-AgenticRAG also supports reasoning, task decomposition, and multi-turn search interactions, improving accuracy in complex diagnostic scenarios. Experiments show that Patho-AgenticRAG significantly outperforms existing multimodal models in complex pathology tasks like multiple-choice diagnosis and visual question answering. Our project is available at the Patho-AgenticRAG repository: https://github.com/Wenchuan-Zhang/Patho-AgenticRAG.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
