EndoGaussian：用于动态内窥镜场景重建的实时高斯泼溅
EndoGaussian: Real-time Gaussian Splatting for Dynamic Endoscopic Scene Reconstruction
AI总结：
提出基于3DGS的实时内窥镜场景重建框架EndoGaussian，通过HGI整体初始化和SGT时空跟踪策略解决组织形变问题，实现195 FPS实时渲染与高质量重建，显著优于先前SOTA。
摘要：
从内窥镜视频中重建可变形组织在许多下游手术应用中至关重要。然而，现有方法渲染速度慢，极大限制了其实际应用。本文提出 EndoGaussian，一个基于 3D Gaussian Splatting (3DGS) 的实时内窥镜场景重建框架。通过集成高效的高斯表示和高度优化的渲染引擎，本框架将渲染速度显著提升至实时水平。为使 3DGS 适应内窥镜场景，我们提出 Holistic Gaussian Initialization (HGI) 和 Spatio-temporal Gaussian Tracking (SGT) 两种策略，分别处理非平凡的高斯初始化和组织形变问题。在 HGI 中，利用近期深度估计模型预测输入双目/单目图像序列的深度图，据此对像素进行重投影并组合以实现整体初始化。在 SGT 中，提出使用由高效编码体素和轻量级形变解码器组成的形变场来建模表面动态，从而以极小的训练和渲染负担实现高斯跟踪。在公开数据集上的实验证明了本方法在多方面优于先前 SOTA，包括更快的渲染速度（195 FPS 实时，100倍提升）、更好的渲染质量（37.848 PSNR）和更少的训练开销（2分钟/场景内），在术中手术应用中展现出重大前景。代码可在 https://yifliu3.github.io/EndoGaussian/ 获取。
英文摘要：
Reconstructing deformable tissues from endoscopic videos is essential in many downstream surgical applications. However, existing methods suffer from slow rendering speed, greatly limiting their practical use. In this paper, we introduce EndoGaussian, a real-time endoscopic scene reconstruction framework built on 3D Gaussian Splatting (3DGS). By integrating the efficient Gaussian representation and highly-optimized rendering engine, our framework significantly boosts the rendering speed to a real-time level. To adapt 3DGS for endoscopic scenes, we propose two strategies, Holistic Gaussian Initialization (HGI) and Spatio-temporal Gaussian Tracking (SGT), to handle the non-trivial Gaussian initialization and tissue deformation problems, respectively. In HGI, we leverage recent depth estimation models to predict depth maps of input binocular/monocular image sequences, based on which pixels are re-projected and combined for holistic initialization. In SPT, we propose to model surface dynamics using a deformation field, which is composed of an efficient encoding voxel and a lightweight deformation decoder, allowing for Gaussian tracking with minor training and rendering burden. Experiments on public datasets demonstrate our efficacy against prior SOTAs in many aspects, including better rendering speed (195 FPS real-time, 100$\times$ gain), better rendering quality (37.848 PSNR), and less training overhead (within 2 min/scene), showing significant promise for intraoperative surgery applications. Code is available at: \url{https://yifliu3.github.io/EndoGaussian/}.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
