arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.01597cs.CVcs.GR
收藏

端到端率失真优化的3D高斯表示

End-to-End Rate-Distortion Optimized 3D Gaussian Representation

University of Science and Technology of China(中国科学技术大学);Microsoft Research Asia(微软亚洲研究院);The University of Adelaide(阿德莱德大学)

Henan Wang, Hanxin Zhu, Tianyu He, Runsen Feng, Jiajun Deng, Jiang Bian, Zhibo Chen

AI总结:

本文提出RDO-Gaussian,将紧凑3D高斯学习建模为端到端率失真优化问题,通过动态剪枝和熵约束向量量化同时优化码率与失真,实现灵活连续码率控制,并将3D高斯大小压缩超过40倍。

摘要:

3D高斯泼溅(3DGS)已成为在3D表示和图像渲染方面具有显著潜力的新兴技术。然而,3DGS巨大的存储开销严重阻碍了其实际应用。在本工作中,我们将紧凑3D高斯学习建模为端到端的率失真优化(RDO)问题,并提出RDO-Gaussian,能够实现灵活且连续的码率控制。RDO-Gaussian解决了当前方案中存在的两个主要问题:1)不同于先前在固定失真下最小化码率的工作,我们引入了动态剪枝和熵约束向量量化(ECVQ),同时优化码率和失真。2)先前的工作对每个高斯的颜色同等对待,而我们用可学习数量的参数对不同区域和材质的颜色进行建模。我们在真实场景和合成场景上验证了我们的方法,结果表明RDO-Gaussian将3D高斯的大小大幅减小超过40倍,并在率失真性能上超越了现有方法。

英文摘要:

3D Gaussian Splatting (3DGS) has become an emerging technique with remarkable potential in 3D representation and image rendering. However, the substantial storage overhead of 3DGS significantly impedes its practical applications. In this work, we formulate the compact 3D Gaussian learning as an end-to-end Rate-Distortion Optimization (RDO) problem and propose RDO-Gaussian that can achieve flexible and continuous rate control. RDO-Gaussian addresses two main issues that exist in current schemes: 1) Different from prior endeavors that minimize the rate under the fixed distortion, we introduce dynamic pruning and entropy-constrained vector quantization (ECVQ) that optimize the rate and distortion at the same time. 2) Previous works treat the colors of each Gaussian equally, while we model the colors of different regions and materials with learnable numbers of parameters. We verify our method on both real and synthetic scenes, showcasing that RDO-Gaussian greatly reduces the size of 3D Gaussian over 40x, and surpasses existing methods in rate-distortion performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑