发表机构
Duke University（杜克大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对手术AR虚拟内容遮挡手术器械的问题，提出结合VLM与分割推理的延迟感知流水线，构建伪AR基准，实现87.43%准确率、479 ms延迟，较云端基线降延迟62.90%。
AI 中文摘要
手术增强现实（AR）可通过在手术工作空间上叠加虚拟标注、器械提示和操作流程信息提供情境化引导，但虚拟内容可能遮挡手术器械等与任务相关的真实信息，干扰用户在时间敏感的手术任务中的感知。本文研究手术AR的视觉遮挡检测，提出一种延迟感知流水线，结合基于视觉语言模型（VLM）的手术对象识别与基于分割的遮挡推理。为降低推理开销，系统采用级联的小到大VLM架构，包含分割引导的早退出和基于注意力的视觉令牌剪枝：当小型VLM对关键对象的预测得到分割一致性支持时，处理简单帧；困难帧则将剪枝后的视觉令牌转发给大型VLM。我们通过在手术器械图像上叠加虚拟内容并标注其是否遮挡任务相关器械，构建了伪AR手术遮挡检测基准。评估结果显示，该系统的遮挡检测准确率达87.43%，平均端到端延迟为479 ms，相比云端大型模型基线延迟降低62.90%。这些结果证明了手术AR的延迟感知遮挡检测的可行性，并为动态手术视频、多对象场景及临床落地AR引导内容的未来工作提供了动力。
英文摘要
Surgical augmented reality (AR) can provide contextual guidance by overlaying virtual annotations, tool cues, and procedural information onto the surgical workspace. However, the virtual content may obstruct task-relevant real-world information, such as surgical instruments, and interfere with users' perception during time-sensitive surgical tasks. In this paper, we investigate visual obstruction detection for surgical AR and present a latency-aware pipeline that combines vision-language model (VLM)-based surgical-object recognition with segmentation-based obstruction reasoning. To reduce inference overhead, the system adopts a cascaded small-to-large VLM architecture with segmentation-guided early exiting and attention-based visual token pruning. The small VLM handles easy frames when its key-object prediction is supported by segmentation consistency, while difficult frames are forwarded to a large VLM with pruned visual tokens. We construct a pseudo-AR surgical obstruction detection benchmark by overlaying virtual content onto surgical-tool images and labeling whether the virtual content obstructs task-relevant instruments. Evaluation results show that the proposed system achieves 87.43% obstruction detection accuracy with an average end-to-end latency of 479 ms, reducing latency by 62.90% compared with a cloud large-model baseline. These results demonstrate the feasibility of latency-aware obstruction detection for surgical AR and motivate future work on dynamic surgical videos, multi-object scenes, and clinically grounded AR guidance content.
CommentsISMAR 2026 Mecidal Workshop
