arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2506.19291cs.CV

HoliGS:用于具身视角合成的整体高斯泼溅

HoliGS: Holistic Gaussian Splatting for Embodied View Synthesis

CMU(卡内基梅隆大学);ETH Zurich(苏黎世联邦理工学院);UC San Diego(圣地亚哥大学);UC Merced(默塞德大学);Insta360(Insta360公司);Google DeepMind(谷歌DeepMind)

Xiaoyuan Wang, Yizhou Zhao, Botao Ye, Xiaojun Shan, Weijie Lyu, Lu Qi, Kelvin C. K. Chan, Yinxiao Li, Ming-Hsuan Yang

AI总结:

提出HoliGS框架,利用可逆高斯泼溅变形网络分层重建大规模动态环境,在长单目RGB视频的具身视角合成中实现高质量渲染并大幅降低训练与渲染时间。

摘要:

我们提出了HoliGS,一种新型可变形高斯泼溅框架,旨在解决从长单目RGB视频中进行具身视角合成的问题。与先前的4D高斯泼溅和动态NeRF管线在分钟级长视频捕获中面临训练开销过大的问题不同,我们的方法利用可逆高斯泼溅变形网络,准确重建大规模动态环境。具体而言,我们将每个场景分解为静态背景和随时间变化的物体,两者均由学习到的高斯基元表示,这些基元通过可逆神经流经历全局刚性变换、骨架驱动的关节运动以及细微的非刚性变形。这种分层变形策略通过将高斯附着于完整的标准前景形状上(例如,以自我为中心或第三人称跟随),实现了从各种具身相机轨迹进行鲁棒的自由视点新视角渲染,这可能涉及大量视角变化和多个参与者之间的交互。我们的实验表明,HoliGS在具有挑战性的数据集上实现了卓越的重建质量,同时与最先进的单目可变形NeRF相比,显著减少了训练和渲染时间。这些结果突显了一种适用于真实世界场景下EVS的实用且可扩展的解决方案。源代码将公开发布。

英文摘要:

We propose HoliGS, a novel deformable Gaussian splatting framework that addresses embodied view synthesis from long monocular RGB videos. Unlike prior 4D Gaussian splatting and dynamic NeRF pipelines, which struggle with training overhead in minute-long captures, our method leverages invertible Gaussian Splatting deformation networks to reconstruct large-scale, dynamic environments accurately. Specifically, we decompose each scene into a static background plus time-varying objects, each represented by learned Gaussian primitives undergoing global rigid transformations, skeleton-driven articulation, and subtle non-rigid deformations via an invertible neural flow. This hierarchical warping strategy enables robust free-viewpoint novel-view rendering from various embodied camera trajectories by attaching Gaussians to a complete canonical foreground shape (\eg, egocentric or third-person follow), which may involve substantial viewpoint changes and interactions between multiple actors. Our experiments demonstrate that \ourmethod~ achieves superior reconstruction quality on challenging datasets while significantly reducing both training and rendering time compared to state-of-the-art monocular deformable NeRFs. These results highlight a practical and scalable solution for EVS in real-world scenarios. The source code will be released.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑