arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2610.21895cs.CV
收藏

SurgStreamer:基于结构化状态与可复用推理层的主动、请求驱动型手术视频流监控系统

SurgStreamer: proactive, request-driven monitoring of surgical video streams with structured state and a reusable reasoning layer

Vanderbilt University(范德堡大学)

Bowen Xiang, Jon S. Heiselman

AI总结:

SurgStreamer将手术视频的感知与推理分离,在腹腔镜胆囊切除术测试中宏事件F1达0.598,外部中心性能提升,新手术流程需校准阈值,验证了分离设计的有效性。

摘要:

手术团队可从能监控视频中特定事件的助手处获益,例如“告诉我在Calot三角解剖期间施夹钳何时出现”,并回答已发生事件的相关问题。我们将该任务形式化为具有精确语义与因果警报决策的请求驱动型事件检测任务,并提出SurgStreamer。它将特定手术流程的感知(将1 FPS的视频流映射为工具、动作与阶段的结构化状态)与流程无关的推理(基于该状态和事件记忆评估形式化监控条件,可采用精确规则或通过指针而非名称处理实体的小型控制器)分离开来。我们采用金标准条件评估该监控核心:在预注册的密封测试集(12例腹腔镜胆囊切除术)上,SurgStreamer在完整手术流程上的宏事件F1值达0.598;在所有方法均仅能使用的相同75秒窗口上,其得分达0.677。经测试的流视觉语言模型OneStreamer-4B的配置(包括在SurgStreamer状态或带有适配视觉编码器的视频上微调的配置)得分最高为0.540(Holm校正后p≤0.012)。在三个外部中心,仅重新训练感知模块即可将未见过的手术流程的性能从0.239提升至0.470。在新手术流程(垂体手术)上,基于胆囊切除术训练的控制器权重在金标准状态下与领域内控制器表现相当(0.947 vs 0.938);在预测状态下,除非重新校准警报阈值,否则其未达到预注册的非劣效性边际(-0.024 [-0.040, -0.008])。结果支持将感知与推理分离的设计,并指出分布偏移下的感知是主要局限。

英文摘要:

Surgical teams could benefit from assistants that watch the video for requested events, such as "tell me when the clip applier appears during Calot triangle dissection", and answer questions about what has already happened. We formalise this task as request-driven event detection with exact semantics and causal alert decisions, and present SurgStreamer. It separates procedure-specific perception, which maps the 1-FPS stream to a structured state over tools, actions and phases, from procedure-agnostic reasoning, which evaluates a formal monitoring condition over this state and an event memory, either with exact rules or with a small controller that addresses entities through pointers rather than names. We evaluate this monitoring core with gold conditions. On a pre-registered, sealed test set of 12 laparoscopic cholecystectomies, SurgStreamer reached a macro event-F1 of 0.598 over full procedures. On identical 75-s windows with only the window available to every method, it scored 0.677. The tested configurations of the streaming vision-language model OneStreamer-4B, including ones fine-tuned on SurgStreamer's state or on video with an adapted vision encoder, scored at most 0.540 (Holm-adjusted p <= 0.012). At three external centres, re-training only perception raised performance on held-out procedures from 0.239 to 0.470. On a new procedure, pituitary surgery, controller weights trained on cholecystectomy matched an in-domain controller on oracle state (0.947 vs 0.938). On predicted state they missed the pre-registered non-inferiority margin (-0.024 [-0.040, -0.008]) unless the alert thresholds were re-calibrated. The results support separating perception from reasoning and point to perception under distribution shift as the main limitation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑