arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2011.02112cs.ROcs.LG
收藏

使用具有视觉和机器人状态的深度学习进行机器人辅助手术中的力估计

Toward Force Estimation in Robot-Assisted Surgery using Deep Learning with Vision and Robot State

Stanford University(斯坦福大学);Intuitive Surgical Inc.(直觉外科公司)

Zonghe Chua, Anthony M. Jarc, Allison M. Okamura

摘要:

在遥控机器人辅助手术期间的相互作用力的知识可以用于使力反馈给人类操作者并评估组织处理技能。然而,在端部执行器处的直接力感测是具有挑战性的,因为它需要生物相容的、可灭菌的和具有成本效益的传感器。使用卷积神经网络的基于视觉的深度学习是提供有用的力估计的一种有前途的方法,尽管关于新场景的推广和实时推理仍然存在问题。我们提出了一个力估计神经网络,使用RGB图像和机器人状态作为输入。使用自我收集的数据集,我们将网络与仅包含单一输入类型的变体进行了比较,并评估了它们如何推广到新的视点,工作空间位置,材料和工具。我们发现,基于视觉的网络对视点的变化很敏感,而仅状态网络对工作空间的变化很鲁棒。具有状态和视觉输入的网络对于看不见的工具具有最高的准确性,并且对视点的变化具有适度的鲁棒性。通过特征去除研究,我们发现仅使用位置特征比仅使用力特征作为输入产生更好的准确性。具有状态和视觉输入的网络在准确性上优于基于物理的基线模型。它表现出相当的准确性,但比基线递归神经网络更快的计算时间,使其更适合实时应用。

英文摘要:

Knowledge of interaction forces during teleoperated robot-assisted surgery could be used to enable force feedback to human operators and evaluate tissue handling skill. However, direct force sensing at the end-effector is challenging because it requires biocompatible, sterilizable, and cost-effective sensors. Vision-based deep learning using convolutional neural networks is a promising approach for providing useful force estimates, though questions remain about generalization to new scenarios and real-time inference. We present a force estimation neural network that uses RGB images and robot state as inputs. Using a self-collected dataset, we compared the network to variants that included only a single input type, and evaluated how they generalized to new viewpoints, workspace positions, materials, and tools. We found that vision-based networks were sensitive to shifts in viewpoints, while state-only networks were robust to changes in workspace. The network with both state and vision inputs had the highest accuracy for an unseen tool, and was moderately robust to changes in viewpoints. Through feature removal studies, we found that using only position features produced better accuracy than using only force features as input. The network with both state and vision inputs outperformed a physics-based baseline model in accuracy. It showed comparable accuracy but faster computation times than a baseline recurrent neural network, making it better suited for real-time applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑