arXiv 2602.174732026-02-19cs.CV

在任意摄像机运动下进行4D单目手术重建

4D Monocular Surgical Reconstruction under Arbitrary Camera Motions

Department of Mechanical and Automation Engineering（机械与自动化工程系）；T Stone Robotics Institute, The Chinese University of Hong Kong, Hong Kong（T Stone机器人研究院，香港中文大学，香港）；Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang, China（沈阳自动化研究所，中国科学院，沈阳，中国）；State Key Laboratory of Robotics and Intelligent Systems, Shenyang, China（机器人与智能系统国家重点实验室，中国，沈阳）；School of Integrated Circuits, Shanghai Jiao Tong University, Shanghai, China（集成电路学院，上海交通大学，中国，上海）；School of Automation and Intelligent Sensing, Shanghai Jiao Tong University, Shanghai, China（自动化与智能感知学院，上海交通大学，中国，上海）；Key Laboratory of System Control and Information Processing, Ministry of Education of China, Shanghai, China（系统控制与信息处理重点实验室，中国教育部，中国，上海）

Jiwei Shan, Zeyu Cai, Cheng-Tai Hsieh, Yirui Li, Hao Liu, Lijun Han, Hesheng Wang, Shing Shin Cheng

备注：: Due to the limitation "The abstract field cannot be longer than 1,920 characters", the abstract here is shorter than that in the PDF file Subjects

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出Local-EndoGS框架，通过局部可变形场景模型和多视图几何优化，实现任意摄像机运动下的高质量4D单目手术重建。

摘要：

从内窥镜视频中重建可变形手术场景具有挑战性且在临床中非常重要。最近基于隐式神经表示或3D高斯散射的最新方法取得了显著进展。然而，大多数方法都是为具有固定内窥镜视角的可变形场景设计的，并且依赖立体深度先验或准确的结构从运动进行初始化和优化，限制了它们处理具有大摄像机运动的单目序列的能力。为了解决这个问题，我们提出了Local-EndoGS，一种高质量的单目内窥镜序列4D重建框架，适用于任意摄像机运动。Local-EndoGS引入了一种渐进的、基于窗口的全局表示，将局部可变形场景模型分配给每个观察到的窗口，使其能够扩展到具有显著运动的长序列。为了克服没有立体深度或准确结构从运动的不可靠初始化问题，我们设计了一种粗到细的策略，整合多视图几何、跨窗口信息和单目深度先验，为优化提供稳健的基础。我们进一步结合长距离2D像素轨迹约束和物理运动先验，以提高变形的合理性。在三个具有可变形场景和不同摄像机运动的公开内窥镜数据集上的实验表明，Local-EndoGS在外观质量和几何方面始终优于最新方法。消融研究验证了我们关键设计的有效性。代码将在接受时发布在：https://github.com/IRMVLab/Local-EndoGS。

英文摘要：

Reconstructing deformable surgical scenes from endoscopic videos is challenging and clinically important. Recent state-of-the-art methods based on implicit neural representations or 3D Gaussian splatting have made notable progress. However, most are designed for deformable scenes with fixed endoscope viewpoints and rely on stereo depth priors or accurate structure-from-motion for initialization and optimization, limiting their ability to handle monocular sequences with large camera motion in real clinical settings. To address this, we propose Local-EndoGS, a high-quality 4D reconstruction framework for monocular endoscopic sequences with arbitrary camera motion. Local-EndoGS introduces a progressive, window-based global representation that allocates local deformable scene models to each observed window, enabling scalability to long sequences with substantial motion. To overcome unreliable initialization without stereo depth or accurate structure-from-motion, we design a coarse-to-fine strategy integrating multi-view geometry, cross-window information, and monocular depth priors, providing a robust foundation for optimization. We further incorporate long-range 2D pixel trajectory constraints and physical motion priors to improve deformation plausibility. Experiments on three public endoscopic datasets with deformable scenes and varying camera motions show that Local-EndoGS consistently outperforms state-of-the-art methods in appearance quality and geometry. Ablation studies validate the effectiveness of our key designs. Code will be released upon acceptance at: https://github.com/IRMVLab/Local-EndoGS.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。