SurgFlow：面向手术机器人操作的三维物体中心接触流
SurgFlow: 3D Object-Centric Contact Flow for Surgical Robot Manipulation
浏览论文内容
中文总结 AI 辅助
SurgFlow提出从立体手术视频中学习三维物体中心接触流，无需动作标签即可预测轨迹和接触点，在dVRK上优于基线并零样本迁移至人形机器人，成功率高达85%。
中文摘要 AI 辅助
配对视频-动作演示能够实现自主手术行为，但此类数据稀缺：机器人仅执行约1%的手术，而纯视频数据却十分丰富。学习三维物体流提供了一种与具体执行器无关的方式来利用视频数据，但仅凭流只能指定物体应如何移动，而不能指定工具应在何处以及何时与其接触，这一区别在手术中至关重要。我们引入了SurgFlow，这是一个无需动作标签即可从立体手术视频中学习三维物体中心接触流的框架。对于每个物体点，它预测未来的三维轨迹和接触分数。我们通过三维跟踪和工具-物体接近度提取目标，训练一个流匹配生成器来预测这些目标，并利用预测的接触来触发抓取和释放，同时根据流优化末端执行器运动。在达芬奇研究套件（dVRK）上，SurgFlow在组织牵拉、双手显露、针拾取和交接等39项阶段评估中成功完成37项，优于在相同数据上训练的有或无动作标签的基线方法。零样本迁移到基于人形机器人的腹腔镜机器人，在相似和新型相机视角下分别实现了85%和70%的平均成功率。
英文摘要
Paired video-action demonstrations enable autonomous surgical behavior, but such data is scarce: robots perform roughly 1% of surgeries, while video-only data is abundant. Learning 3D object flow offers an embodiment-agnostic way to utilize video data, but flow alone specifies how an object should move, not where and when the tool should engage it, a distinction that is critical in surgery. We introduce SurgFlow, a framework that learns 3D Object-Centric Contact Flow from stereo surgical video without action labels. For each object point, it predicts a future 3D trajectory and contact scores. We extract targets via 3D tracking and tool-object proximity, train a flow matching generator to predict them, and use predicted contact to trigger grasp and release while optimizing end effector motion from flow. On the da Vinci Research Kit (dVRK), SurgFlow succeeds in 37 of 39 stage evaluations across tissue retraction, bimanual reveal, needle pickup, and handover, outperforming baselines trained on equal data with or without action labels. Zero-shot transfer to a humanoid-based laparoscopic robot achieves 85% and 70% average success under similar and novel camera viewpoints, respectively.
