arXiv 2510.045872025-10-13cs.CV

Pathology-CoT：从专家全切片图像诊断行为中学习视觉思维链智能体

Pathology-CoT: Learning Visual Chain-of-Thought Agent from Expert Whole Slide Image Diagnosis Behavior

Department of Pathology and Laboratory Medicine, University of Pennsylvania（病理学与实验室医学系，宾夕法尼亚大学）；Department of Biostatistics, Epidemiology & Informatics, University of Pennsylvania（生物统计学、流行病学与信息学系，宾夕法尼亚大学）；Department of Pathology, University of California at San Francisco（病理学系，加州大学旧金山分校）；Department of Pathology, Stanford University（病理学系，斯坦福大学）；Department of Electrical and System Engineering, University of Pennsylvania（电气与系统工程系，宾夕法尼亚大学）

Sheng Wang, Ruiming Wu, Charles Herndon, Yihang Liu, Shunsuke Koga, Jeanne Shen, Zhi Huang

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

提出Pathology-CoT框架，通过记录专家诊断行为构建思维链数据集，训练Pathology-o3智能体，在淋巴结转移检测中超越OpenAI o3，实现高召回率。

摘要：

诊断全切片图像是一个交互式的、多阶段的过程，涉及改变放大倍数和在视野之间移动。尽管最近出现的病理学基础模型展现了优越的性能，但能够决定下一步检查哪个视野、调整放大倍数并提供可解释诊断的实用智能体系统仍然缺乏。这一局限性主要受数据瓶颈制约：可扩展的、与临床对齐的专家观察行为监督是隐性且基于经验的，既未记录在教科书或互联网中，因此也缺失于大语言模型的训练中。在此，我们引入了一个框架，旨在通过三项关键突破来解决这一挑战。首先，AI会话记录器与标准全切片图像查看器无缝集成，能够无干扰地记录常规导航，并将查看器日志转换为标准化的行为命令和边界框。其次，轻量级的人机协同审查将AI起草的行为命令理由转化为Pathology-CoT数据集，这是一种配对的“看哪里”和“为何重要”的形式，使得标注速度比手动构建此类思维链数据集快六倍。利用这些行为数据，我们构建了Pathology-o3，一个两阶段智能体，首先提出重要的感兴趣区域，然后执行行为引导的推理。在胃肠淋巴结转移检测任务中，我们的方法在斯坦福医学中心的内部验证上达到了100%的召回率，在瑞典的独立外部验证上达到了97.6%的召回率，超过了最先进的OpenAI o3模型，并在不同骨干网络上具有泛化能力。据我们所知，Pathology-CoT构成了病理学中首批基于行为的智能体系统之一。将日常查看器日志转化为可扩展的、专家验证的监督，我们的框架使智能体病理学变得实用，并为与人类对齐的、可升级的临床AI建立了一条路径。

英文摘要：

Diagnosing a whole-slide image is an interactive, multi-stage process of changing magnification and moving between fields. Although recent pathology foundation models demonstrated superior performances, practical agentic systems that decide what field to examine next, adjust magnification, and deliver explainable diagnoses are still lacking. Such limitation is largely bottlenecked by data: scalable, clinically aligned supervision of expert viewing behavior that is tacit and experience-based, not documented in textbooks or internet, and therefore absent from LLM training. Here we introduce a framework designed to address this challenge through three key breakthroughs. First, the AI Session Recorder seamlessly integrates with standard whole-slide image viewers to unobtrusively record routine navigation and convert the viewer logs into standardized behavioral commands and bounding boxes. Second, a lightweight human-in-the-loop review turns AI-drafted rationales for behavioral commands into the Pathology-CoT dataset, a form of paired "where to look" and "why it matters", enabling six-fold faster labeling compared to manual constructing such Chain-of-Thought dataset. Using this behavioral data, we build Pathology-o3, a two-stage agent that first proposes important ROIs and then performs behavior-guided reasoning. On the gastrointestinal lymph-node metastasis detection task, our method achieved 100 recall on the internal validation from Stanford Medicine and 97.6 recall on an independent external validation from Sweden, exceeding the state-of-the-art OpenAI o3 model and generalizing across backbones. To our knowledge, Pathology-CoT constitutes one of the first behavior-grounded agentic systems in pathology. Turning everyday viewer logs into scalable, expert-validated supervision, our framework makes agentic pathology practical and establishes a path to human-aligned, upgradeable clinical AI.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。