发表机构
Columbia University（哥伦比亚大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文提出一种从单目视频重建手术器械运动学的网络，结合DINOv3和SAM特征，通过可微加权最小二乘拟合轨迹，在Open-H基准上显著降低路径长度误差并提升运动分割精度。
AI 中文摘要
机器人手术的客观评估使用器械运动学，当仅有视频可用时必须重建这些运动学。我们提出了一种运动学重建网络，用于从单目视频中估计器械位置、方向和钳口角度。我们的视觉表示结合了冻结DINOv3特征的全局注意力池化与来自微调SAM 3.1掩膜的器械标志点局部池化。我们的共享Transformer编码器和时间卷积头将该表示与掩膜几何、单目深度以及来自特定手臂多层回归网络的视觉状态估计相整合。我们的位置分支分别预测位移大小和方向，以保留行进距离。我们通过可微加权最小二乘法将轨迹拟合到预测的状态观测和运动增量，将四元数观测相对于累积预测旋转表示，以获得二次方向目标。我们在2,802个Open-H episodes上评估重建效果。与主Open-H基准上的LiveMAE相比，我们的方法将路径长度平均绝对误差从0.45厘米降低到0.34厘米，并将运动分割的时间平均精度从44.54%提高到54.44%。
英文摘要
Objective assessment of robotic surgery uses instrument kinematics, which must be reconstructed when only video is available. We introduce a kinematic reconstruction network for estimating instrument position, orientation and jaw angle from monocular video. Our visual representation combines global attention pooling of frozen DINOv3 features with local pooling at instrument landmarks from fine-tuned SAM 3.1 masks. Our shared Transformer encoder and temporal convolutional heads integrate this representation with mask geometry, monocular depth and visual state estimates from arm-specific multilayer regression networks. Our position branch predicts displacement magnitude and direction separately to preserve traveled distance. We fit trajectories to predicted state observations and motion increments by differentiable weighted least squares, expressing quaternion observations relative to cumulative predicted rotations to obtain a quadratic orientation objective. We evaluate reconstruction across 2,802 Open-H episodes. Compared with LiveMAE on the main Open-H benchmark, our method reduces path-length mean absolute error from 0.45 to 0.34\,cm and increases temporal mean average precision for motion segmentation from 44.54\% to 54.44\%.
