arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.22792cs.RO
收藏

Instrument-Splatting++: 向可控手术工具数字双胞胎迈进的高斯点云技术

Instrument-Splatting++: Towards Controllable Surgical Instrument Digital Twin Using Gaussian Splatting

Department of Biomedical Engineering, National University of Singapore (NUS)（生物医学工程系，新加坡国立大学）；Department of Electrical and Computer Engineering, The University of British Columbia（电气与计算机工程系，不列颠哥伦比亚大学）

Shuojue Yang, Zijian Wu, Chengjiaao Liao, Qian Li, Daiyun Shen, Chang Han Low, Septimiu E. Salcudean, Yueming Jin

备注：
10 pages, 9 figures
阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出Instrument-Splatting++框架，通过高斯点云技术实现高保真可控的手术工具数字双胞胎，利用CAD先验知识和语义感知姿态估计提升重建精度和纹理学习效果。

摘要：

高质量且可控的手术工具数字双胞胎对于机器人辅助手术中的Real2Sim至关重要，因为它们能够实现逼真的模拟、合成数据生成和感知学习。我们提出了Instrument-Splatting++，一种单目3D高斯点云（3DGS）框架，能够以高保真度重建手术工具为完全可控的高斯资产。我们的流程首先进行部分几何预训练，将CAD先验注入高斯原始体并赋予部分感知语义渲染。基于预训练模型，我们提出语义感知姿态估计和跟踪（SAPET）方法，从未姿态内窥镜视频中恢复每帧6自由度姿态和关节角度，其中训练纯合成语义的夹具尖端网络提供鲁棒监督，松散正则化抑制奇异关节运动。最后，我们引入鲁棒纹理学习（RTL），交替姿态优化和鲁棒外观优化，缓解纹理学习中的姿态噪声。所提框架能够从未姿态视频中进行姿态估计并学习逼真的纹理。我们在EndoVis17/18、SAR-RARP和内部数据集的序列上验证了方法，显示在光度质量和几何准确性上优于现有最先进基线。我们进一步展示了下游关键点检测任务，其中未见姿态数据增强从我们的可控仪器高斯资产中提升了性能。

英文摘要：

High-quality and controllable digital twins of surgical instruments are critical for Real2Sim in robot-assisted surgery, as they enable realistic simulation, synthetic data generation, and perception learning under novel poses. We present Instrument-Splatting++, a monocular 3D Gaussian Splatting (3DGS) framework that reconstructs surgical instruments as a fully controllable Gaussian asset with high fidelity. Our pipeline starts with part-wise geometry pretraining that injects CAD priors into Gaussian primitives and equips the representation with part-aware semantic rendering. Built on the pretrained model, we propose a semantics-aware pose estimation and tracking (SAPET) method to recover per-frame 6-DoF pose and joint angles from unposed endoscopic videos, where a gripper-tip network trained purely from synthetic semantics provides robust supervision and a loose regularization suppresses singular articulations. Finally, we introduce Robust Texture Learning (RTL), which alternates pose refinement and robust appearance optimization, mitigating pose noise during texture learning. The proposed framework can perform pose estimation and learn realistic texture from unposed videos. We validate our method on sequences extracted from EndoVis17/18, SAR-RARP, and an in-house dataset, showing superior photometric quality and improved geometric accuracy over state-of-the-art baselines. We further demonstrate a downstream keypoint detection task where unseen-pose data augmentation from our controllable instrument Gaussian improves performance.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑