arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2606.08093cs.AI
收藏

面向证据基础计算病理学的多模态智能体协同助手

A Multimodal Agentic Pathology Co-pilot via Evidence Grounded Reasoning

Department of Computer Science and Engineering, Hong Kong University of Science and Technology(香港科技大学计算机科学与工程系);Department of Pathology, Nanfang Hospital, Southern Medical University(南方医科大学南芳医院病理科);Department of Pathology, School of Basic Medical Sciences, Southern Medical University(南方医科大学基础医学学院病理科);Department of Anatomical and Cellular Pathology, Chinese University of Hong Kong(香港中文大学解剖与细胞病理学系);Guangdong Provincial Key Laboratory of Molecular Tumor Pathology(广东省分子肿瘤病理学重点实验室);Jinfeng Laboratory(锦风实验室);Department of Chemical and Biological Engineering, Hong Kong University of Science and Technology(香港科技大学化学与生物工程系);Division of Life Science, Hong Kong University of Science and Technology(香港科技大学生命科学系);State Key Laboratory of Nervous System Disorders, The Hong Kong University of Science and Technology(香港科技大学神经系统疾病国家重点实验室);HKUST Shenzhen-Hong Kong Collaborative Innovation Research Institute, The Hong Kong University of Science and Technology(香港科技大学深圳-香港协同创新研究院)
Zhe Xu, Zhengyu Zhang, Zhiyuan Cai, Jiahao Xu, Yijie Lin, Ziyi Liu, Junlin Hou, Hongyi Wang, Yuxiang Nie, Yihui Wang, Jiabo Ma, Ling Liang, Yingxue Xu, Zhengrui… 展开作者

Zhe Xu, Zhengyu Zhang, Zhiyuan Cai, Jiahao Xu, Yijie Lin, Ziyi Liu, Junlin Hou, Hongyi Wang, Yuxiang Nie, Yihui Wang, Jiabo Ma, Ling Liang, Yingxue Xu, Zhengrui Guo, Guanghao Wu, Danyi Li, Ziqi Zhou, Donglin Tan, Zhijian Cen, Ying Tan, Xiaolin Liu, Qi Xie, Xiaoying Tang, Xi Peng, Cheng Deng, Lijuan Qu, Ronald Cheong Kin Chan, Li Liang, Hao Chen

AI总结:

提出PathPocket,一种多模态AI协同助手,通过构建包含11万文档的病理证据语料库和455万实体的超图,实现基于证据的病理诊断,在20万真实案例上超越现有方法。

摘要:

病理学是现代医学的基石,准确的决策高度依赖于循证实践。虽然人工智能有潜力改变临床工作流程,但AI与循证医学的结合仍未被充分探索,现有的初步尝试仅限于纯文本的通用医学。在这项工作中,我们提出了PathPocket,一种专门为证据基础病理学设计的多模态AI智能体协同助手。我们构建了迄今为止最全面的病理证据语料库,包含约110,472份公开和授权文档,这些文档按照从临床指南到专家意见的严格证据层级进行结构化组织。在这个精心分级的基础上,我们构建了一个大规模多模态病理超图,包含超过455万个实体和710万个关系。作为强大的知识引擎,该超图为协作式多智能体推理框架提供了可追溯的证据,该框架集成了输入理解、证据检索、过滤和诊断生成。这使得PathPocket能够无缝解决广泛的临床任务,从纯文本查询到涉及感兴趣区域和千兆像素全切片图像的复杂多模态诊断。我们在一个包含超过20万真实案例的多维基准测试上严格评估了该系统,其性能显著优于现有最先进方法。至关重要的是,广泛的用户研究表明,PathPocket显著提高了病理学家的诊断准确性和信心。通过将病理学解释直接基于可验证的文献,PathPocket为未来证据基础的计算病理学提供了实用且可扩展的解决方案。

英文摘要:

Pathology is the cornerstone of modern medicine, where accurate decision-making relies heavily on evidence-based practices. While artificial intelligence (AI) has the potential to transform clinical workflows, the intersection of AI and evidence-based medicine remains under-explored, with primitive attempts restricted to text-only general medicine. In this work, we present PathPocket, a multimodal AI agentic co-pilot designed specifically for evidence grounded pathology. We construct the most comprehensive pathology evidence corpus to date, encompassing approximately 110,472 public and authorized documents structured across a rigorous hierarchy of evidence from clinical guideline to expert opinion. From this meticulously graded foundation, we build a large-scale multimodal pathology hypergraph containing over 4.55 million entities and 7.10 million relations. Serving as a robust knowledge engine, this hypergraph provides traceable evidence for a collaborative multi-agent reasoning framework integrating input understanding, evidence retrieval, filtering, and diagnosis generation. This enables PathPocket to seamlessly resolve a wide spectrum of clinical tasks, ranging from text-only queries to complex multimodal diagnostics involving region-of-interest (ROI) and gigapixel whole-slide images (WSIs). We rigorously evaluate the system on a multidimensional benchmark of over 200,000 real-world cases, where it significantly outperforms existing state-of-the-arts. Crucially, extensive user studies demonstrate that PathPocket substantially improves the diagnostic accuracy and confidence of pathologists. By directly grounding pathology interpretations in verifiable literature, PathPocket offers a practical and scalable solution for the future of evidence grounded computational pathology.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑