评估用于自主计算病理学的智能控制框架系统
Evaluating Agentic Harness Systems for Autonomous Computational Pathology
AI总结:
研究将高级病理学分析目标转化为可执行工作流,贡献 ACP-Bench 框架,评估 41 个病理学工作流任务、9 个模型和 3 个控制框架组,通过多方面评估为智能系统实现病理学工作流提供标准。
摘要:
自主计算病理学(ACP)将高级病理学分析目标转化为可执行、可追溯且临床受限的工作流程。实现此功能需要使通用智能控制框架系统适应病理学特定任务、工具、证据标准和临床主张界限。我们贡献了 ACP-Bench,一个从计算病理学支持将现有控制框架系统适配到 ACP 工作流能力的框架。ACP-Bench 评估 41 个病理学工作流任务,包括 24 个生物标志物、7 个形态学和 10 个预后任务,跨越 6 个身体系统组和 9 个终点类别。该基准评估 9 个模型和 3 个控制框架组(Claude Code、Codex 和 Open Code),产生 369 条完整轨迹。ACP-Bench 通过专家裁决的过程审核、诊断评估和病理学家验证的安全审查,在工作流执行、诊断性能和临床边界对齐方面评估每条轨迹。在评估的系统中,工作流启动、任务解释和诊断报告比工具绑定执行、结果绑定和反思性工作流修订更成熟,正式的端到端完成仍然很少见。ACP-Bench 提供了一个可重复使用的标准,用于在声称可靠的临床自主性之前审核智能系统是否可以将病理学工作流投入运行。
英文摘要:
Autonomous computational pathology (ACP) converts high-level pathology analysis goals into executable, traceable and clinically bounded workflows. Realizing this capability requires adapting general agentic harness systems to pathology-specific tasks, tools, evidence standards and clinical claim boundaries. We contribute ACP-Bench, a framework that adapts existing harness systems from computational pathology support toward ACP workflow capability. ACP-Bench evaluates 41 pathology workflow tasks, including 24 biomarker, 7 morphology and 10 prognosis tasks spanning 6 body-system groups and 9 endpoint families. The benchmark evaluates 9 models and 3 harness groups (Claude Code, Codex and Open Code), yielding 369 complete trajectories. ACP-Bench evaluates each trajectory across workflow execution, diagnostic performance and clinical-boundary alignment, combining expert-adjudicated process audits, diagnostic assessment and pathologist-validated safety review. Across evaluated systems, workflow initiation, task interpretation and diagnostic reporting were more mature than tool-bound execution, result binding and reflective workflow revision, and formal end-to-end completion remained rare. ACP-Bench provides a reusable standard for auditing whether agentic systems can operationalize pathology workflows before claims of reliable clinical autonomy.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。