arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.05166cs.CV
收藏

CD-NGP:一种用于动态场景的快速可扩展持续表示

CD-NGP: A Fast Scalable Continual Representation for Dynamic Scenes

Dept. of Automation, Shanghai Jiao Tong University(自动化系,上海交通大学)

Zhenhuan Liu, Shuai Liu, Zhiwei Ning, Jie Yang, Yifan Zuo, Yuming Fang, Wei Liu

AI总结:

本文提出CD-NGP持续学习框架,通过参数重用和时空哈希编码实现动态场景的高效新视角合成,并构建长时长多视角视频数据集,显著降低内存与带宽需求。

摘要:

动态场景中的新视角合成(NVS)在内存消耗、模型复杂度、训练效率和渲染质量方面面临持续挑战。离线方法提供高保真度,但存在高内存占用和有限的可扩展性问题,而在线方法通常以质量换取速度和紧凑性。我们提出持续动态神经图形基元(CD-NGP),这是一个持续学习框架,通过参数重用来降低内存开销并增强可扩展性。为避免动态场景中的特征干扰并提高渲染质量,我们的方法结合了空间和时间哈希编码,以紧凑地表示场景结构和运动模式。我们还引入了一个新的数据集,包含多视角、长时长(>1200帧)的视频,同时具有刚性和非刚性运动,这在现有基准中是不存在的。CD-NGP在公开数据集和我们的长视频数据集上进行了评估,展示了优越的可扩展性和重建质量。在DyNeRF上,它显著将训练内存使用降低到<14GB,并且流式带宽仅需0.4MB/帧——远低于大多数在线基线。

英文摘要:

Novel view synthesis (NVS) in dynamic scenes faces persistent challenges in memory consumption, model complexity, training efficiency, and rendering quality. Offline methods offer high fidelity but suffer from high memory usage and limited scalability, while online approaches often trade quality for speed and compactness. We propose Continual Dynamic Neural Graphics Primitives (CD-NGP), a continual learning framework that reduces memory overhead and enhances scalability through parameter reuse. To avoid feature interference in dynamic scenes and improve rendering quality, our method combines spatial and temporal hash encodings, which compactly represent scene structures and motion patterns. We also introduce a new dataset comprising multi-view, long-duration ($>1200$ frames) videos with both rigid and non-rigid motion, which is not found in existing benchmarks. CD-NGP is evaluated on public datasets and our long video dataset, demonstrating superior scalability and reconstruction quality. It significantly reduces training memory usage to <14GB and requires only 0.4MB/frame in streaming bandwidth on DyNeRF -- substantially lower than most online baselines.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑