3D Gaussian Splatting (3DGS) has become an emerging technique with remarkable potential in 3D representation and image rendering. However, the substantial storage overhead of 3DGS significantly impedes its practical applications. In this work, we formulate the compact 3D Gaussian learning as an end-to-end Rate-Distortion Optimization (RDO) problem and propose RDO-Gaussian that can achieve flexible and continuous rate control. RDO-Gaussian addresses two main issues that exist in current schemes: 1) Different from prior endeavors that minimize the rate under the fixed distortion, we introduce dynamic pruning and entropy-constrained vector quantization (ECVQ) that optimize the rate and distortion at the same time. 2) Previous works treat the colors of each Gaussian equally, while we model the colors of different regions and materials with learnable numbers of parameters. We verify our method on both real and synthetic scenes, showcasing that RDO-Gaussian greatly reduces the size of 3D Gaussian over 40x, and surpasses existing methods in rate-distortion performance.
Rate-distortion Optimized Point Cloud Preprocessing for Geometry-based Point Cloud Compression
School of Telecommunications Engineering, Xidian University(电子科技大学电信工程学院);School of Electronics and Information, Northwestern Polytechnical University(西北工业大学电子信息技术学院);School of Engineering, Royal Melbourne Institute of Technology(皇家墨尔本理工学院工程学院)
RDCompGS: Rate-Distortion Optimization for Compression of 3D Gaussian Splatting with Image Codecs
School of Electronics and Information, Northwestern Polytechnical University(西北工业大学电子与信息工程学院);Department of Computer Science, City University of Hong Kong(香港城市大学计算机系);School of Telecommunication Engineering, Xidian University(西安电子科技大学通信工程学院)
Rate-Distortion Optimized Skip Coding of Region Adaptive Hierarchical Transform Coefficients for MPEG G-PCC
School of Control Science and Engineering, Shandong University(山东大学控制科学与工程学院);School of Telecommunications Engineering, Xidian University(西安电子科技大学通信工程学院)
Distortion-Corrected Diffusion MRI Using Rotated-View EPI and Joint Field-Map/Image Estimation with Gaussian Primitives
Chair for AI in Healthcare and Medicine, Technical University of Munich (TUM) and TUM University Hospital, and the School of Computation, Information and Technology, TUM(慕尼黑工业大学(TUM)医疗与医学人工智能教席、TUM大学医院、计算信息与技术学院);Neuroimaging Technology Research Center, Department of Radiology and Biomedical Imaging, University of California, San Francisco(加州大学旧金山分校放射与生物医学影像系神经影像技术研究中心);Innovation Academy for Precision Measurement Science and Technology, Chinese Academy of Sciences(中国科学院精密测量科学与技术创新研究院);Department of Radiology, Stanford University(斯坦福大学放射学系);Munich Center for Machine Learning (MCML)(慕尼黑机器学习中心(MCML));Department of Computing, Imperial College London(伦敦帝国理工学院计算系)