MesonGS: Post-training Compression of 3D Gaussians via Efficient Attribute Transformation
MMLab, The Chinese University of Hong Kong（香港中文大学MMLab实验室）；The University of Texas at Austin（德克萨斯大学奥斯汀分校）；SIGS & TBSI, Tsinghua University（清华大学深圳国际研究生院与清华-伯克利深圳学院）；Peng Cheng Laboratory（鹏城实验室）
3D Gaussian Splatting（3D高斯泼溅）在新视角合成中展现出优异的质量与速度。然而，3D高斯的巨大文件尺寸给传输和存储带来了挑战。现有工作设计紧凑模型来替代3D高斯的庞大体积与属性，并通过密集训练来蒸馏信息，这些工作需要大量训练时间，给实际部署带来了极大阻碍。为此，我们提出MesonGS，一种用于3D高斯训练后压缩的编解码器。首先，我们提出一种同时考虑视角相关和视角无关因素的度量准则，用于评估每个高斯点对渲染输出的影响，从而移除不重要的点。随后，我们通过两种变换降低属性的熵，与后续的熵编码技术配合以提升文件压缩率。具体而言，我们首先用欧拉角替换旋转四元数；接着对关键属性应用区域自适应分层变换以降低熵。最后，我们采用更细粒度的量化以避免过度信息损失。此外，我们设计了一套精心调配的微调方案来恢复质量。大量实验表明，MesonGS在保持具有竞争力的质量的同时，显著减小了3D高斯的尺寸。
英文摘要：
3D Gaussian Splatting demonstrates excellent quality and speed in novel view synthesis. Nevertheless, the huge file size of the 3D Gaussians presents challenges for transmission and storage. Current works design compact models to replace the substantial volume and attributes of 3D Gaussians, along with intensive training to distill information. These endeavors demand considerable training time, presenting formidable hurdles for practical deployment. To this end, we propose MesonGS, a codec for post-training compression of 3D Gaussians. Initially, we introduce a measurement criterion that considers both view-dependent and view-independent factors to assess the impact of each Gaussian point on the rendering output, enabling the removal of insignificant points. Subsequently, we decrease the entropy of attributes through two transformations that complement subsequent entropy coding techniques to enhance the file compression rate. More specifically, we first replace rotation quaternions with Euler angles; then, we apply region adaptive hierarchical transform to key attributes to reduce entropy. Lastly, we adopt finer-grained quantization to avoid excessive information loss. Moreover, a well-crafted finetune scheme is devised to restore quality. Extensive experiments demonstrate that MesonGS significantly reduces the size of 3D Gaussians while preserving competitive quality.
MesonGS++: Post-training Compression of 3D Gaussian Splatting with Hyperparameter Searching
Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；The Hong Kong University of Science and Technology（香港科技大学）；Harbin Institute of Technology（哈尔滨工业大学）；The Chinese University of Hong Kong（香港中文大学）；The University of Texas at Austin（德克萨斯大学奥斯汀分校）；Shenzhen Technology University（深圳技术大学）；Xiamen University（厦门大学）；Simon Fraser University（西蒙弗雷泽大学）；Jiangxing Intelligence Inc.（江兴智能有限公司）
Sun Yat-sen University（中山大学）；The Shenzhen Campus of Sun Yat-sen University（中山大学深圳校区）；University of Oxford（牛津大学）；Information Engineering University（信息工程大学）；Shanghai Jiaotong University（上海交通大学）；National University of Defense Technology（国防科技大学）
We study the problem of attribute compression for large-scale unstructured 3D point clouds. Through an in-depth exploration of the relationships between different encoding steps and different attribute channels, we introduce a deep compression network, termed 3DAC, to explicitly compress the attributes of 3D point clouds and reduce storage usage in this paper. Specifically, the point cloud attributes such as color and reflectance are firstly converted to transform coefficients. We then propose a deep entropy model to model the probabilities of these coefficients by considering information hidd…
A Benchmark for Gaussian Splatting Compression and Quality Assessment Study
School of Science and Engineering, University of Missouri - Kansas City（密苏里大学堪萨斯城分校科学与工程学院）；Cooperative Medianet Innovation Center, Shanghai Jiaotong University（上海交通大学合作 medianet 创新中心）
In point cloud compression, exploiting temporal redundancy for inter predictive coding is challenging because of the irregular geometry. This paper proposes an efficient block-based inter-coding scheme for color attribute compression. The scheme includes integer-precision motion estimation and an adaptive graph based in-loop filtering scheme for improved attribute prediction. The proposed block-based motion estimation scheme consists of an initial motion search that exploits geometric and color attributes, followed by a motion refinement that only minimizes color prediction error. To further i…
Sun Yat-sen University（中山大学）；University of Oxford（牛津大学）；Shanghai Jiaotong University（上海交通大学）
With the development of the 3D data acquisition facilities, the increasing scale of acquired 3D point clouds poses a challenge to the existing data compression techniques. Although promising performance has been achieved in static point cloud compression, it remains under-explored and challenging to leverage temporal correlations within a point cloud sequence for effective dynamic point cloud compression. In this paper, we study the attribute (e.g., color) compression of dynamic point clouds and present a learning-based framework, termed 4DAC. To reduce temporal redundancy within data, we firs…
University of Science and Technology of China（中国科学技术大学）
Geometry-based point cloud compression (G-PCC) can achieve remarkable compression efficiency for point clouds. However, it still leads to serious attribute compression artifacts, especially under low bitrate scenarios. In this paper, we propose a Multi-Scale Graph Attention Network (MS-GAT) to remove the artifacts of point cloud attributes compressed by G-PCC. We first construct a graph based on point cloud geometry coordinates and then use the Chebyshev graph convolutions to extract features of point cloud attributes. Considering that one point may be correlated with points both near and far …
Hangzhou Normal University（杭州师范大学）；Nanjing University（南京大学）
A learning-based adaptive loop filter is developed for the Geometry-based Point Cloud Compression (G-PCC) standard to reduce attribute compression artifacts. The proposed method first generates multiple Most-Probable Sample Offsets (MPSOs) as potential compression distortion approximations, and then linearly weights them for artifact mitigation. As such, we drive the filtered reconstruction as close to the uncompressed PCA as possible. To this end, we devise a Compression Artifact Reduction Network (CARNet) which consists of two consecutive processing phases: MPSOs derivation and MPSOs combina…
DALD-PCAC: Density-Adaptive Learning Descriptor for Point Cloud Lossless Attribute Compression
City University of Hong Kong（香港城市大学）；Peking University Shenzhen Graduate School（北京大学深圳研究生院）；University of Missouri-Kansas City（密苏里大学-堪萨斯城分校）；Tencent Media Laboratory（腾讯媒体实验室）
Deep-JGAC: End-to-End Deep Joint Geometry and Attribute Compression for Dense Colored Point Clouds
School of Electronics and Communication Engineering, Shenzhen Campus, Sun Yat-Sen University（中山大学深圳校区电子与通信工程学院）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Department of Electrical and Computer Engineering, University of Southern California（南加州大学电子与计算机工程系）
Efficient and Generic Point Model for Lossless Point Cloud Attribute Compression
College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics（南京航空航天大学计算机科学与技术学院）；School of Electronic Science and Engineering, Nanjing University（南京大学电子科学与工程学院）
Att2CPC: Attention-Guided Lossy Attribute Compression of Point Clouds
College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics（南京航空航天大学计算机科学与技术学院）；School of Computing and Mathematics, Charles Sturt University（查尔斯斯特大学计算与数学学院）
UGAE: Unified Geometry and Attribute Enhancement for G-PCC Compressed Point Clouds
School of Control Science and Engineering, Shandong University（控制科学与工程学院，山东大学）；Key Laboratory of Machine Intelligence and System Control, Ministry of Education（机器智能与系统控制重点实验室，教育部）；School of Engineering and Sustainable Development, De Montfort University（工程与可持续发展学院，德蒙福特大学）；Institute of NUIST-MetaX for Graphics Processing Unit, Nanjing University of Information Science and Technology（信息科学技术大学NUIST-MetaX图形处理单元研究所）
Peng Cheng Lab（鹏程实验室）；Peking University, Shenzhen Graduate School（北京大学深圳研究生院）；University of Southern California（南加州大学）
Inspired by the recently proposed successive subspace learning (SSL) principles, we develop a successive subspace graph transform (SSGT) to address point cloud attribute compression in this work. The octree geometry structure is utilized to partition the point cloud, where every node of the octree represents a point cloud subspace with a certain spatial size. We design a weighted graph with self-loop to describe the subspace and define a graph Fourier transform based on the normalized graph Laplacian. The transforms are applied to large point clouds from the leaf nodes to the root node of the …
Existing techniques to compress point cloud attributes leverage either geometric or video-based compression tools. We explore a radically different approach inspired by recent advances in point cloud representation learning. Point clouds can be interpreted as 2D manifolds in 3D space. Specifically, we fold a 2D grid onto a point cloud and we map attributes from the point cloud onto the folded 2D grid using a novel optimized mapping method. This mapping results in an image, which opens a way to apply existing image processing techniques on point cloud attributes. However, as this mapping proces…
Rendering-Oriented 3D Point Cloud Attribute Compression using Sparse Tensor-based Transformer
School of Telecommunication Engineering, Xidian University（西安电子科技大学通信工程学院）；City University of Hong Kong（香港城市大学）；School of Electronics and Information, Northwestern Polytechnical University（西北工业大学电子信息学院）；School of Engineering, Royal Melbourne Institute of Technology（皇家墨尔本理工大学工程学院）
Packet-Loss Robust 3D Gaussian Compression via Atomic Packaging and GNN-based Error Concealment
Central South University Changsha China；Malanshan Audio \& Video Laboratory Changsha China；Central South University；Malanshan Audio \& Video Laboratory
RDCompGS: Rate-Distortion Optimization for Compression of 3D Gaussian Splatting with Image Codecs
School of Electronics and Information, Northwestern Polytechnical University（西北工业大学电子与信息工程学院）；Department of Computer Science, City University of Hong Kong（香港城市大学计算机系）；School of Telecommunication Engineering, Xidian University（西安电子科技大学通信工程学院）
Nanjing University（南京大学）；Hangzhou Normal University（杭州师范大学）；University of Missouri（密苏里大学）
Recent years have witnessed the growth of point cloud based applications because of its realistic and fine-grained representation of 3D objects and scenes. However, it is a challenging problem to compress sparse, unstructured, and high-precision 3D points for efficient communication. In this paper, leveraging the sparsity nature of point cloud, we propose a multiscale end-to-end learning framework which hierarchically reconstructs the 3D Point Cloud Geometry (PCG) via progressive re-sampling. The framework is developed on top of a sparse convolution based autoencoder for point cloud compressio…
SPAC: Sampling-based Progressive Attribute Compression for Dense Point Clouds
School of Control Science and Engineering, Shandong University（山东大学控制科学与工程学院）；School of Engineering and Sustainable Development, De Montfort University（德蒙福特大学工程与可持续发展学院）；School of Data Science, Lingnan University（岭南大学数据科学学院）
Towards Practical Compression of 3D Gaussian Splatting
Sun Yat-sen University（中山大学）；Pengcheng Laboratory（鹏城实验室）；Academy of Broadcasting Science, National Radio and Television Administration（国家广播电视总局广播电视科学研究院）；Peking University Shenzhen Graduate School（北京大学深圳研究生院）
College of Information Science and Electronic Engineering, Zhejiang University（浙江大学信息科学与电子工程学院）；Department of Electrical and Electronic Engineering, The University of Hong Kong（香港大学电子与电气工程系）；College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算机与数据科学学院）
Ecole Polytechnique Fédérale de Lausanne (EPFL)（洛桑联邦理工学院）；Microsoft Research（微软研究院）
This paper addresses the problem of compression of 3D point cloud sequences that are characterized by moving 3D positions and color attributes. As temporally successive point cloud frames are similar, motion estimation is key to effective compression of these sequences. It however remains a challenging problem as the point cloud frames have varying numbers of points without explicit correspondence information. We represent the time-varying geometry of these sequences with a set of graphs, and consider 3D positions and color attributes of the points clouds as signals on the vertices of the grap…
