arXiv 2602.133322026-02-11cs.CVcs.AI

MedScope：通过粗到细的工具调用激励“通过视频思考”以进行临床推理

MedScope: Incentivizing "Think with Videos" for Clinical Reasoning via Coarse-to-Fine Tool Calling

College of Health Science and Technology, Shanghai Jiao Tong University School of Medicine, Shanghai, China（上海交通大学医学院健康科学与技术学院）；Fudan University, Shanghai, China（复旦大学）；Shanghai Innovation Institute, Shanghai, China（上海创新研究院）；Tsinghua University, Beijing, China（清华大学）；Shanghai Artificial Intelligence Laboratory, Shanghai, China（上海人工智能实验室）；The Chinese University of Hong Kong, Hong Kong, China（香港中文大学）；Ruijin Hospital, Shanghai Jiaotong University, Shanghai, China（上海交通大学瑞金医院）

Wenjie Li, Yujie Zhang, Haoran Sun, Xingqi He, Hongcheng Gao, Chenglong Ma, Ming Hu, Guankun Wang, Shiyi Yao, Renhao Yang, Hongliang Ren, Lei Wang, Junjun He, Yankai Jiang

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

MedScope通过粗到细的工具调用机制，提升临床视频推理的准确性与可信度，实现基于时间局部化视觉证据的医疗AI代理。

摘要：

长形式的临床视频是视觉证据驱动决策的核心，对于手术机器人等应用日益重要。然而，当前的多模态大语言模型通常使用被动采样或弱相关检查来处理视频，这限制了它们通过时间针对性的证据迭代定位、验证和证明预测的能力。为填补这一差距，我们提出了MedScope，一个进行粗到细证据寻找的临床视频推理模型。通过将中间推理与针对目标的工具调用和对检索观察的验证相结合，MedScope产生更加准确和可信的预测，这些预测明确基于时间局部化的视觉证据。为了解决缺乏高保真监督的问题，我们构建了ClinVideoSuite，一个以证据为中心、细粒度的临床视频集。然后我们通过Grounding-Aware Group Relative Policy Optimization（GA-GRPO）优化MedScope，该方法直接通过与接地对齐的奖励和证据加权优势来强化工具使用。在完整的和细粒度的视频理解基准测试中，MedScope在领域内和领域外评估中均取得了最先进的性能。我们的方法照亮了通往能够真正“通过视频思考”的医疗AI代理的道路。我们将发布我们的代码、模型和数据。

英文摘要：

Long-form clinical videos are central to visual evidence-based decision-making, with growing importance for applications such as surgical robotics and related settings. However, current multimodal large language models typically process videos with passive sampling or weakly grounded inspection, which limits their ability to iteratively locate, verify, and justify predictions with temporally targeted evidence. To close this gap, we propose MedScope, a tool-using clinical video reasoning model that performs coarse-to-fine evidence seeking over long-form procedures. By interleaving intermediate reasoning with targeted tool calls and verification on retrieved observations, MedScope produces more accurate and trustworthy predictions that are explicitly grounded in temporally localized visual evidence. To address the lack of high-fidelity supervision, we build ClinVideoSuite, an evidence-centric, fine-grained clinical video suite. We then optimize MedScope with Grounding-Aware Group Relative Policy Optimization (GA-GRPO), which directly reinforces tool use with grounding-aligned rewards and evidence-weighted advantages. On full and fine-grained video understanding benchmarks, MedScope achieves state-of-the-art performance in both in-domain and out-of-domain evaluations. Our approach illuminates a path toward medical AI agents that can genuinely "think with videos" through tool-integrated reasoning. We will release our code, models, and data.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。