数字孪生驱动的多视角空间感知VR遥操作手术机器人系统
Digital Twin-Driven VR Teleoperation with Multi-View Spatial Perception for Surgical Robots
浏览论文内容
中文总结 AI 辅助
本文提出数字孪生驱动的VR遥操作平台，通过多视角渲染和运动视差改善手术机器人深度感知，利用VR控制器提升控制稳定性，在dVRK上显著优于HoloLens 2基线。
中文摘要 AI 辅助
当前机器人辅助微创手术（RMIS）平台为外科医生提供固定控制台，用于查看立体内窥镜图像并操作患者体内的器械。部分研究者提出使用头戴式显示器（HMD）作为便携式控制台，通过视频穿透渲染内窥镜图像，但与传统固定控制台一样，这种方式将操作者限制在单一内窥镜视角，并限制了深度感知。我们提出了一种数字孪生驱动的虚拟现实（VR）遥操作平台，其中数字孪生通过无标记感知手术环境创建，并流式传输至HMD显示。该平台通过提供多视角渲染和自然运动视差线索，克服了视频穿透的局限性，实现了用户手部姿态与严格器械对齐的解耦。系统利用VR手部控制器来扩大遥操作工作空间，并相比大多数先前系统采用的手部跟踪方法，提高了器械控制的鲁棒性和稳定性。在达芬奇研究套件（dVRK）上进行的一项15名参与者用户研究表明，我们的VR平台显著优于最先进的HoloLens 2混合现实基线，路径长度减少86%，急动度减少95%，同时在所有条件下实现了与传统控制台相当或更优的深度感知置信度。
英文摘要
Current robot-assisted minimally-invasive surgery (RMIS) platforms provide a fixed console for the surgeon to view stereo endoscopic images and teleoperate instruments inside the patient. Several researchers have proposed the use of a head-mounted display (HMD) as a portable console, with video pass-through rendering of the endoscope images which, like the fixed console, restricts the operator to a single endoscopic viewpoint and limits depth perception. We present a digital twin-driven virtual reality (VR) teleoperation platform, where the digital twin is created from markerless perception of the surgical environment and streamed for display on the HMD. This overcomes the limitations of video pass-through by providing multi-view rendering and natural motion-parallax cues, enabling decoupling of the user's hand posture from strict instrument alignment. The system utilizes VR hand controllers to increase the teleoperation workspace and to improve the robustness and stability of instrument control compared to the hand tracking approach adopted by most prior systems. A 15-participant user study on the da Vinci Research Kit (dVRK) shows that our VR platform significantly outperforms a state-of-the-art HoloLens 2 mixed reality baseline, reducing path length by 86% and jerk by 95%, while achieving depth perception confidence comparable to or exceeding the traditional console across all conditions.
发表机构
- Johns Hopkins University（约翰霍普金斯大学）
机构由 AI 辅助整理，请以论文原文为准。
