Point cloud compression is essential to experience volumetric multimedia as it drastically reduces the required streaming data rates. Point attributes, specifically colors, extend the challenge of lossy compression beyond geometric representation to achieving joint reconstruction of texture and geometry. State-of-the-art methods separate geometry and attributes to compress them individually. This comes at a computational cost, requiring an encoder and a decoder for each modality. Additionally, as attribute compression methods require the same geometry for encoding and decoding, the encoder emulates the decoder-side geometry reconstruction as an input step to project and compress the attributes. In this work, we propose to learn joint compression of geometry and attributes using a single, adaptive autoencoder model, embedding both modalities into a unified latent space which is then entropy encoded. Key to the technique is to replace the search for trade-offs between rate, attribute quality and geometry quality, through conditioning the model on the desired qualities of both modalities, bypassing the need for training model ensembles. To differentiate important point cloud regions during encoding or to allow view-dependent compression for user-centered streaming, conditioning is pointwise, which allows for local quality and rate variation. Our evaluation shows comparable performance to state-of-the-art compression methods for geometry and attributes, while reducing complexity compared to related compression methods.
Sun Yat-sen University（中山大学）；The Shenzhen Campus of Sun Yat-sen University（中山大学深圳校区）；University of Oxford（牛津大学）；Information Engineering University（信息工程大学）；Shanghai Jiaotong University（上海交通大学）；National University of Defense Technology（国防科技大学）
We study the problem of attribute compression for large-scale unstructured 3D point clouds. Through an in-depth exploration of the relationships between different encoding steps and different attribute channels, we introduce a deep compression network, termed 3DAC, to explicitly compress the attributes of 3D point clouds and reduce storage usage in this paper. Specifically, the point cloud attributes such as color and reflectance are firstly converted to transform coefficients. We then propose a deep entropy model to model the probabilities of these coefficients by considering information hidd…
In point cloud compression, exploiting temporal redundancy for inter predictive coding is challenging because of the irregular geometry. This paper proposes an efficient block-based inter-coding scheme for color attribute compression. The scheme includes integer-precision motion estimation and an adaptive graph based in-loop filtering scheme for improved attribute prediction. The proposed block-based motion estimation scheme consists of an initial motion search that exploits geometric and color attributes, followed by a motion refinement that only minimizes color prediction error. To further i…
Sun Yat-sen University（中山大学）；University of Oxford（牛津大学）；Shanghai Jiaotong University（上海交通大学）
With the development of the 3D data acquisition facilities, the increasing scale of acquired 3D point clouds poses a challenge to the existing data compression techniques. Although promising performance has been achieved in static point cloud compression, it remains under-explored and challenging to leverage temporal correlations within a point cloud sequence for effective dynamic point cloud compression. In this paper, we study the attribute (e.g., color) compression of dynamic point clouds and present a learning-based framework, termed 4DAC. To reduce temporal redundancy within data, we firs…
University of Science and Technology of China（中国科学技术大学）
Geometry-based point cloud compression (G-PCC) can achieve remarkable compression efficiency for point clouds. However, it still leads to serious attribute compression artifacts, especially under low bitrate scenarios. In this paper, we propose a Multi-Scale Graph Attention Network (MS-GAT) to remove the artifacts of point cloud attributes compressed by G-PCC. We first construct a graph based on point cloud geometry coordinates and then use the Chebyshev graph convolutions to extract features of point cloud attributes. Considering that one point may be correlated with points both near and far …
Hangzhou Normal University（杭州师范大学）；Nanjing University（南京大学）
A learning-based adaptive loop filter is developed for the Geometry-based Point Cloud Compression (G-PCC) standard to reduce attribute compression artifacts. The proposed method first generates multiple Most-Probable Sample Offsets (MPSOs) as potential compression distortion approximations, and then linearly weights them for artifact mitigation. As such, we drive the filtered reconstruction as close to the uncompressed PCA as possible. To this end, we devise a Compression Artifact Reduction Network (CARNet) which consists of two consecutive processing phases: MPSOs derivation and MPSOs combina…
DALD-PCAC: Density-Adaptive Learning Descriptor for Point Cloud Lossless Attribute Compression
City University of Hong Kong（香港城市大学）；Peking University Shenzhen Graduate School（北京大学深圳研究生院）；University of Missouri-Kansas City（密苏里大学-堪萨斯城分校）；Tencent Media Laboratory（腾讯媒体实验室）
Deep-JGAC: End-to-End Deep Joint Geometry and Attribute Compression for Dense Colored Point Clouds
School of Electronics and Communication Engineering, Shenzhen Campus, Sun Yat-Sen University（中山大学深圳校区电子与通信工程学院）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Department of Electrical and Computer Engineering, University of Southern California（南加州大学电子与计算机工程系）
Att2CPC: Attention-Guided Lossy Attribute Compression of Point Clouds
College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics（南京航空航天大学计算机科学与技术学院）；School of Computing and Mathematics, Charles Sturt University（查尔斯斯特大学计算与数学学院）
Existing techniques to compress point cloud attributes leverage either geometric or video-based compression tools. We explore a radically different approach inspired by recent advances in point cloud representation learning. Point clouds can be interpreted as 2D manifolds in 3D space. Specifically, we fold a 2D grid onto a point cloud and we map attributes from the point cloud onto the folded 2D grid using a novel optimized mapping method. This mapping results in an image, which opens a way to apply existing image processing techniques on point cloud attributes. However, as this mapping proces…
Peng Cheng Lab（鹏程实验室）；Peking University, Shenzhen Graduate School（北京大学深圳研究生院）；University of Southern California（南加州大学）
Inspired by the recently proposed successive subspace learning (SSL) principles, we develop a successive subspace graph transform (SSGT) to address point cloud attribute compression in this work. The octree geometry structure is utilized to partition the point cloud, where every node of the octree represents a point cloud subspace with a certain spatial size. We design a weighted graph with self-loop to describe the subspace and define a graph Fourier transform based on the normalized graph Laplacian. The transforms are applied to large point clouds from the leaf nodes to the root node of the …
Efficient and Generic Point Model for Lossless Point Cloud Attribute Compression
College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics（南京航空航天大学计算机科学与技术学院）；School of Electronic Science and Engineering, Nanjing University（南京大学电子科学与工程学院）
Packet-Loss Robust 3D Gaussian Compression via Atomic Packaging and GNN-based Error Concealment
Central South University Changsha China；Malanshan Audio \& Video Laboratory Changsha China；Central South University；Malanshan Audio \& Video Laboratory
Rendering-Oriented 3D Point Cloud Attribute Compression using Sparse Tensor-based Transformer
School of Telecommunication Engineering, Xidian University（西安电子科技大学通信工程学院）；City University of Hong Kong（香港城市大学）；School of Electronics and Information, Northwestern Polytechnical University（西北工业大学电子信息学院）；School of Engineering, Royal Melbourne Institute of Technology（皇家墨尔本理工大学工程学院）
MesonGS: Post-training Compression of 3D Gaussians via Efficient Attribute Transformation
MMLab, The Chinese University of Hong Kong（香港中文大学MMLab实验室）；The University of Texas at Austin（德克萨斯大学奥斯汀分校）；SIGS & TBSI, Tsinghua University（清华大学深圳国际研究生院与清华-伯克利深圳学院）；Peng Cheng Laboratory（鹏城实验室）
Nanjing University（南京大学）；Hangzhou Normal University（杭州师范大学）；University of Missouri（密苏里大学）
Recent years have witnessed the growth of point cloud based applications because of its realistic and fine-grained representation of 3D objects and scenes. However, it is a challenging problem to compress sparse, unstructured, and high-precision 3D points for efficient communication. In this paper, leveraging the sparsity nature of point cloud, we propose a multiscale end-to-end learning framework which hierarchically reconstructs the 3D Point Cloud Geometry (PCG) via progressive re-sampling. The framework is developed on top of a sparse convolution based autoencoder for point cloud compressio…
SPAC: Sampling-based Progressive Attribute Compression for Dense Point Clouds
School of Control Science and Engineering, Shandong University（山东大学控制科学与工程学院）；School of Engineering and Sustainable Development, De Montfort University（德蒙福特大学工程与可持续发展学院）；School of Data Science, Lingnan University（岭南大学数据科学学院）
A Benchmark for Gaussian Splatting Compression and Quality Assessment Study
School of Science and Engineering, University of Missouri - Kansas City（密苏里大学堪萨斯城分校科学与工程学院）；Cooperative Medianet Innovation Center, Shanghai Jiaotong University（上海交通大学合作 medianet 创新中心）
College of Information Science and Electronic Engineering, Zhejiang University（浙江大学信息科学与电子工程学院）；Department of Electrical and Electronic Engineering, The University of Hong Kong（香港大学电子与电气工程系）；College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算机与数据科学学院）
Whiting School of Engineering, Johns Hopkins University（约翰斯·霍普金斯大学怀廷工程学院）
Synthesis of face images from visual attributes is an important problem in computer vision and biometrics due to its applications in law enforcement and entertainment. Recent advances in deep generative networks have made it possible to synthesize high-quality face images from visual attributes. However, existing methods are specifically designed for generating unimodal images (i.e visible faces) from attributes. In this paper, we propose a novel generative adversarial network that simultaneously synthesizes identity preserving multimodal face images (i.e. visible, sketch, thermal, etc.) from …
MISC: Ultra-low Bitrate Image Semantic Compression Driven by Large Multimodal Model
Institute of Image Communication and Network Engineering, Shanghai Jiao Tong University（上海交通大学图像通信与网络工程研究所）；John Hopcroft Center, Shanghai Jiao Tong University（上海交通大学约翰·霍普克罗夫特中心）；S-Lab, Nanyang Technological University（南洋理工大学S-Lab）
Revisiting Multimodal KV Cache Compression: A Frequency-Domain-Guided Outlier-KV-Aware Approach
College of Future Information Technology, Fudan University（未来信息科技学院，复旦大学）；Shanghai Innovation Institute（上海创新研究院）；The Chinese University of Hong Kong（香港中文大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Zhangjiang Laboratory（张江实验室）
Chain-of-Thought Compression Should Not Be Blind: V-Skip for Efficient Multimodal Reasoning via Dual-Path Anchoring
School of Software Engineering, Xi’an Jiaotong University（西安交通大学软件工程学院）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Harbin Institute of Technology, Shenzhen（深圳哈尔滨工业大学）；University of Science and Technology Beijing（北京科技大学）
Task-Oriented Visual Feature Compression via Residual Vector Quantization for Device-Edge Multimodal Inference
Tsinghua University（清华大学）；Beijing National Research Center for Information Science and Technology（北京信息科学与技术国家研究中心）；China Telecom（中国电信）；Institute of Artificial Intelligence (TeleAI), China Telecom（中国电信人工智能研究院（TeleAI））
As LiDAR sensors have become ubiquitous, the need for an efficient LiDAR data compression algorithm has increased. Modern LiDARs produce gigabytes of scan data per hour and are often used in applications with limited compute, bandwidth, and storage resources. We present a fast, lossless compression algorithm for LiDAR range and attribute scan sequences including multiple-return range, signal, reflectivity, and ambient infrared. Our algorithm -- dubbed "Jiffy" -- achieves substantial compression by exploiting spatiotemporal redundancy and sparsity. Speed is accomplished by maximizing use of sin…
A Survey of Token Compression for Efficient Multimodal Large Language Models
Zhejiang University（浙江大学）；Westlake University（西湖大学）；Xiamen University（厦门大学）；National University of Singapore（新加坡国立大学）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；University of Central Florida（佛罗里达大学）；Salesforce AI Research（Salesforce AI研究）；Rice University（德克萨斯大学）
MGIMM: Multi-Granularity Instruction Multimodal Model for Attribute-Guided Remote Sensing Image Detailed Description
National Engineering Research Center for Multimedia Software（国家多媒体软件工程技术研究中心）；School of Computer Science, Wuhan University（武汉大学计算机学院）；Hubei Luojia Laboratory（湖北珞珈实验室）
Can Visual Input Be Compressed? A Visual Token Compression Benchmark for Large Multimodal Models
Shandong University（山东大学）；Shenzhen University of Information Technology（深圳信息职业技术大学）；Tongji University（同济大学）；University of Science and Technology of China（中国科学技术大学）；Sichuan University（四川大学）；University of Notre Dame（圣母大学）；Ludwig Maximilian University of Munich（慕尼黑路德维希-马克西米利安大学）；EB Tech Co., Ltd.（EB科技有限公司）
AdaTok: Adaptive Token Compression with Object-Aware Representations for Efficient Multimodal LLMs
Institute of Medical Technology, Peking University Health Science Center（北京大学医学部医学技术研究所）；Department of Biomedical Engineering, Peking University（北京大学生物医学工程系）；National Biomedical Imaging Center, Peking University（北京大学国家生物医学成像中心）
Lightweight Attribute Localizing Models for Pedestrian Attribute Recognition
Skolkovo Institute of Science and Technology (SKOLTECH)（斯科尔科沃科学技术研究所）；Innopolis University（因诺波利斯大学）；University of Sharjah（沙迦大学）；Advanced Micro Devices (AMD)（超威半导体）；University of Brighton（布莱顿大学）
