Instrument-Splatting：利用高斯泼溅实现手术器械的可控照片级真实感重建
Instrument-Splatting: Controllable Photorealistic Reconstruction of Surgical Instruments Using Gaussian Splatting
AI总结：
提出 Instrument-Splatting，结合几何预训练、正向运动学与语义高斯位姿跟踪，从单目手术视频实现可控、照片级真实的手术器械三维重建。
摘要：
随着外科人工智能（AI）与自主技术的快速发展，Real2Sim 正变得愈发重要。在本研究中，我们提出了一种新颖的 Real2Sim 方法 Instrument-Splatting，该方法利用 3D Gaussian Splatting（三维高斯泼溅），从单目手术视频中实现手术器械的完全可控三维重建。为同时保持高视觉保真度与可操作性，我们引入几何预训练，将高斯点云绑定到带有精确几何先验的部件网格上，并定义正向运动学，使高斯体能够像真实器械一样灵活受控。随后，为处理无相机位姿的视频，我们设计了一种新颖的器械位姿跟踪方法，利用语义嵌入的高斯体，以渲染并比较的方式稳健地优化逐帧器械位姿和关节状态，从而使我们的器械高斯体能够准确学习纹理并实现照片级真实感渲染。我们在 2 个公开发布的手术视频以及 4 个在离体组织和绿幕上采集的视频上验证了该方法。定量与定性评估证明了所提方法的有效性和优越性。
英文摘要：
Real2Sim is becoming increasingly important with the rapid development of surgical artificial intelligence (AI) and autonomy. In this work, we propose a novel Real2Sim methodology, Instrument-Splatting, that leverages 3D Gaussian Splatting to provide fully controllable 3D reconstruction of surgical instruments from monocular surgical videos. To maintain both high visual fidelity and manipulability, we introduce a geometry pre-training to bind Gaussian point clouds on part mesh with accurate geometric priors and define a forward kinematics to control the Gaussians as flexible as real instruments. Afterward, to handle unposed videos, we design a novel instrument pose tracking method leveraging semantics-embedded Gaussians to robustly refine per-frame instrument poses and joint states in a render-and-compare manner, which allows our instrument Gaussian to accurately learn textures and reach photorealistic rendering. We validated our method on 2 publicly released surgical videos and 4 videos collected on ex vivo tissues and green screens. Quantitative and qualitative evaluations demonstrate the effectiveness and superiority of the proposed method.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
