高质量的全新视角合成(NVS)从真实世界视频中至关重要,对于文化遗产保护、数字孪生和沉浸式媒体等应用具有重要意义。然而,真实世界视频通常包含长序列,具有不规则的摄像机轨迹和未知姿态,导致重建过程中出现姿态漂移、特征错位和几何失真。此外,有损压缩会加剧这些问题,通过引入不一致性和逐渐降低几何和渲染质量。尽管最近的研究已经解决了长序列NVS或无姿态重建问题,但压缩感知方法仍集中在特定的伪影或有限场景上,未能充分探索长视频中多样的压缩模式。在本文中,我们提出了CompSplat,一种压缩感知训练框架,该框架明确建模帧级压缩特性,以缓解帧间不一致性和累积的几何误差。CompSplat结合了压缩感知的帧加权和自适应剪枝策略,以增强鲁棒性和几何一致性,特别是在高压缩情况下。在具有挑战性的基准测试中,包括Tanks and Temples、Free和Hike,大量实验表明,CompSplat在渲染质量和姿态准确性方面达到了最先进的水平,显著超越了大多数最近的NVS方法在严苛压缩条件下。
英文摘要:
High-quality novel view synthesis (NVS) from real-world videos is crucial for applications such as cultural heritage preservation, digital twins, and immersive media. However, real-world videos typically contain long sequences with irregular camera trajectories and unknown poses, leading to pose drift, feature misalignment, and geometric distortion during reconstruction. Moreover, lossy compression amplifies these issues by introducing inconsistencies that gradually degrade geometry and rendering quality. While recent studies have addressed either long-sequence NVS or unposed reconstruction, compression-aware approaches still focus on specific artifacts or limited scenarios, leaving diverse compression patterns in long videos insufficiently explored. In this paper, we propose CompSplat, a compression-aware training framework that explicitly models frame-wise compression characteristics to mitigate inter-frame inconsistency and accumulated geometric errors. CompSplat incorporates compression-aware frame weighting and an adaptive pruning strategy to enhance robustness and geometric consistency, particularly under heavy compression. Extensive experiments on challenging benchmarks, including Tanks and Temples, Free, and Hike, demonstrate that CompSplat achieves state-of-the-art rendering quality and pose accuracy, significantly surpassing most recent state-of-the-art NVS approaches under severe compression conditions.
MoE Key Laboratory of Brain-inspired Intelligent Perception and Cognition, University of Science and Technology of China(脑启发式智能感知与认知联合实验室,中国科学技术大学);University of Missouri, Kansas City(密苏里大学,堪萨斯城)
GFix: Perceptually Enhanced Gaussian Splatting Video Compression
Visual Information Laboratory, University of Bristol(布里斯托大学视觉信息实验室);University of Bristol(布里斯托大学);School of Informatics, University of Edinburgh(爱丁堡大学信息学院);Visionular Inc.(Visionular公司)
Nix and Fix: Targeting 1000x Compression of 3D Gaussian Splatting with Diffusion Models
1 Chair of Media Technology, Munich Institute of Robotics;Machine Intelligence School of Computation, Information;Technology Technical University of Munich, 80333 Munich, Germany 2LTCI, T\'el\'ecom Paris, Institut Polytechnique de Paris, France
Improving 3D Gaussian Splatting Compression by Scene-Adaptive Lattice Vector Quantization
Department of Electrical & Computer Engineering, McMaster University(麦卡特尼大学电气与计算机工程系);School of Computing and Artificial Intelligence, Southwest Jiaotong University(西南交通大学计算机与人工智能学院);School of Computer Science and Technology, Tongji University(同济大学计算机科学与技术学院)
Towards Practical Compression of 3D Gaussian Splatting
Sun Yat-sen University(中山大学);Pengcheng Laboratory(鹏城实验室);Academy of Broadcasting Science, National Radio and Television Administration(国家广播电视总局广播电视科学研究院);Peking University Shenzhen Graduate School(北京大学深圳研究生院)
A Benchmark for Gaussian Splatting Compression and Quality Assessment Study
School of Science and Engineering, University of Missouri - Kansas City(密苏里大学堪萨斯城分校科学与工程学院);Cooperative Medianet Innovation Center, Shanghai Jiaotong University(上海交通大学合作 medianet 创新中心)
SparseStreet: Sparse Gaussian Splatting for Real-Time Street Scene Simulation
Peking University(北京大学);Chinese Academy of Sciences(中国科学院);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Autonomous Driving Development, NIO(蔚来自动驾驶开发)
Light4GS: Lightweight Compact 4D Gaussian Splatting Generation via Context Model
School of Information Science and Electronic Engineering, Shanghai Jiao Tong University(信息科学与电子工程学院,上海交通大学);School of Science and Engineering, University of Missouri–Kansas City(科学与工程学院,密苏里大学-堪萨斯城分校);University of Chinese Academy of Sciences(中国科学院大学)
BitC-3DGS: High-Capacity 3D Gaussian Splatting Watermarking via Bit Compression
School of Cyber Science and Engineering, Southeast University(东南大学计算机科学与工程学院);Lee Kong Chian School of Medicine, Nanyang Technological University(南洋理工大学李科金医学院);College of Electronic Engineer, National University of Defense and Technology(国防科技大学电子工程学院);Institute of AI for Industries, Chinese Academy of Sciences(中国科学院人工智能产业研究所);School of Computer Science and Engineering, Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications, Ministry of Education, Southeast University(东南大学计算机科学与工程学院,教育部新一代人工智能技术及其交叉应用重点实验室);Department of Computer and Information Science, University of Macau(澳门大学计算机与信息科学系);Faculty of Science and Technology, UOW College Hong Kong(UOW学院香港科技学院理学院);Department of Computer Science and Engineering, The Hong Kong University of Science and Technology(香港科学与技术大学计算机科学与工程学院)
VEDAL: Variational Error-Driven Asynchronous Learning for 3D Gaussian Splatting Pruning
Guangdong University of Technology(广东工业大学);Huizhou Boluo Power Supply Bureau, Guangdong Power Grid Co., Ltd.(惠州市博罗供电局,广东电网有限责任公司);Shenzhen Polytechnic University(深圳职业技术大学);School of Computer Science and Engineering, Huizhou University(惠州市大学计算机科学与工程学院)
提出VEDAL框架,通过变分自由能最小化、预测误差门控机制和变分不确定性头实现3D高斯泼溅的高效剪枝,在5.2倍压缩下仅损失0.31 dB PSNR。
GaussianPile: A Unified Sparse Gaussian Splatting Framework for Slice-based Volumetric Reconstruction
Tsinghua University(清华大学);Zhongguancun Academy(中关村学院);Beijing Normal University(北京师范大学);Nankai University(南开大学);Beijing Institute of Technology(北京理工大学);Beihang University(北航)