发表机构
University of California San Diego（加州大学圣迭戈分校）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对手术机器人器械跟踪中真实标注昂贵的问题，提出一种跟踪器引导的自训练框架，利用机器人关节状态和不确定性感知EKF及RTS平滑器生成伪标签，微调基于合成图像预训练的特征检测器，在真实视频上提升了关键点指标并优于先前方法。
AI 中文摘要
高效且鲁棒的手术机器人器械跟踪对于机器人辅助微创手术至关重要，然而由于手术场景的复杂性和手术器械的非传统几何形状，这一任务仍然具有挑战性。基于关键点的方法效率较高，但其性能依赖于可靠的特征检测。使用真实世界监督来改进这些检测器是困难的，因为大规模获取准确的真实世界标注成本高昂。为解决这一局限，我们引入了一种跟踪器引导的自训练框架，该框架将基于合成图像预训练的模型适应到未标注的真实世界视频。给定测量的机器人关节状态，一种不确定性感知的扩展卡尔曼滤波器（EKF）通过将投影的模型特征与检测到的关键点、轴边界和掩膜衍生线索进行比较，递归地修正器械姿态和可观测关节角度。随后，一个RTS平滑器对生成的轨迹进行细化，该轨迹被投影为伪标签，用于微调特征检测器，而无需繁琐的姿态标注。在真实世界视频上的实验表明，自训练在所有评估的关键点指标上均带来一致的改进，且所得模型在准确性和运行时间上均优于先前方法。代码和数据将在发表后发布。
英文摘要
Efficient and robust tracking of surgical robotic instruments is important for robot-assisted minimally invasive surgery, yet remains challenging due to the complexity of surgical scenes and the unconventional geometry of surgical instruments. Keypoint-based approaches are efficient, but their performance depends on reliable feature detection. Improving these detectors with real-world supervision is difficult because accurate real-world annotations are costly to obtain at scale. To address this limitation, we introduce a tracker-guided self-training framework that adapts a model pretrained on synthetic images to unlabeled real-world videos. Given measured robot joint states, an uncertainty-aware EKF recursively corrects the instrument pose and the observable joint angles by comparing projected model features with detected keypoints, shaft boundaries, and mask-derived cues. An RTS smoother subsequently refines the resulting trajectory, which is projected into pseudo-labels for fine-tuning the feature detector without laborious pose annotations. Experiments on real-world videos demonstrate consistent improvements from self-training across all evaluated keypoint metrics, and the resulting model outperforms prior approaches in both accuracy and runtime. The code and data will be released upon publication.
