弥合视觉与语言鸿沟以实现鲁棒的上下文感知手术点跟踪：VL-SurgPT数据集与基准
Bridging Vision and Language for Robust Context-Aware Surgical Point Tracking: The VL-SurgPT Dataset and Benchmark
AI总结：
针对手术点跟踪因复杂视觉条件及缺乏语义上下文导致的挑战，本文构建首个多模态VL-SurgPT数据集，提出文本引导的TG-SurgPT方法，显著提升恶劣场景下跟踪精度与可靠性，为上下文感知手术跟踪系统提供基准支持。
摘要：
手术环境中的精确点跟踪因复杂视觉条件（包括烟雾遮挡、镜面反射和组织变形）仍具挑战性。现有手术跟踪数据集虽提供坐标信息，但缺乏理解跟踪失败机制所需的语义上下文。我们引入VL-SurgPT，这是首个将视觉跟踪与手术场景中点状态文本描述相连接的大规模多模态数据集。该数据集包含908个在体视频片段，其中754个用于组织跟踪（涵盖5种挑战性场景的17171个标注点），154个用于器械跟踪（覆盖7种器械类型，含详细关键点标注）。我们使用8种最先进的跟踪方法建立综合基准，并提出TG-SurgPT——一种利用语义描述提升视觉挑战性条件下鲁棒性的文本引导跟踪方法。实验结果表明，融入点状态信息显著提高了跟踪精度和可靠性，尤其在传统纯视觉方法难以应对的恶劣视觉场景中。通过弥合视觉与语言模态，VL-SurgPT为开发上下文感知跟踪系统提供了支持，这对推进计算机辅助手术应用至关重要，此类应用即使在具有挑战性的术中条件下也能保持性能。
英文摘要：
Accurate point tracking in surgical environments remains challenging due to complex visual conditions, including smoke occlusion, specular reflections, and tissue deformation. While existing surgical tracking datasets provide coordinate information, they lack the semantic context necessary to understand tracking failure mechanisms. We introduce VL-SurgPT, the first large-scale multimodal dataset that bridges visual tracking with textual descriptions of point status in surgical scenes. The dataset comprises 908 in vivo video clips, including 754 for tissue tracking (17,171 annotated points across five challenging scenarios) and 154 for instrument tracking (covering seven instrument types with detailed keypoint annotations). We establish comprehensive benchmarks using eight state-of-the-art tracking methods and propose TG-SurgPT, a text-guided tracking approach that leverages semantic descriptions to improve robustness in visually challenging conditions. Experimental results demonstrate that incorporating point status information significantly improves tracking accuracy and reliability, particularly in adverse visual scenarios where conventional vision-only methods struggle. By bridging visual and linguistic modalities, VL-SurgPT enables the development of context-aware tracking systems crucial for advancing computer-assisted surgery applications that can maintain performance even under challenging intraoperative conditions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
