MEGA: Memory-Efficient 4D Gaussian Splatting for Dynamic Scenes
iComAI Lab, The Hong Kong University of Science and Technology（香港科技大学iComAI实验室）；Skywork AI；The Chinese University of Hong Kong（香港中文大学）；National University of Singapore（新加坡国立大学）；Institute for AI Industry Research (AIR), Tsinghua University（清华大学人工智能产业研究院）
Xinjie Zhang, Zhening Liu, Yifan Zhang, Xingtong Ge, Dailan He, Tongda Xu, Yan Wang, Zehong Lin, Shuicheng Yan, Jun Zhang
4D高斯泼溅（4DGS）近年来已成为一种有前景的技术，能够高保真地捕捉复杂动态3D场景。它利用4D高斯表示和GPU友好的光栅化器，实现了快速渲染速度。尽管具有这些优势，4DGS仍面临显著挑战，尤其是需要数百万个4D高斯，每个高斯都带有大量关联属性，导致巨大的内存和存储成本。本文介绍了一种面向4DGS的内存高效框架。我们通过将颜色属性分解为仅含3个参数的逐高斯直接颜色分量和一个共享的轻量级交流颜色预测器来简化颜色属性。该方法消除了对球谐系数的需求，在经典4DGS中球谐系数通常涉及多达144个参数，从而创建了一种内存高效的4D高斯表示。此外，我们引入了一种熵约束高斯变形技术，该技术使用变形场扩展每个高斯的作用范围，并集成基于不透明度的熵损失以限制高斯数量，从而迫使我们的模型使用尽可能少的高斯来良好地拟合动态场景。通过简单的半精度存储和zip压缩，与原始4DGS相比，我们的框架在Technicolor和Neural 3D Video数据集上分别实现了约190倍和125倍的存储缩减。同时，它保持了可比的渲染速度和场景表示质量，为该领域设立了新标准。代码可在https://github.com/Xinjie-Q/MEGA获取。
英文摘要：
4D Gaussian Splatting (4DGS) has recently emerged as a promising technique for capturing complex dynamic 3D scenes with high fidelity. It utilizes a 4D Gaussian representation and a GPU-friendly rasterizer, enabling rapid rendering speeds. Despite its advantages, 4DGS faces significant challenges, notably the requirement of millions of 4D Gaussians, each with extensive associated attributes, leading to substantial memory and storage cost. This paper introduces a memory-efficient framework for 4DGS. We streamline the color attribute by decomposing it into a per-Gaussian direct color component with only 3 parameters and a shared lightweight alternating current color predictor. This approach eliminates the need for spherical harmonics coefficients, which typically involve up to 144 parameters in classic 4DGS, thereby creating a memory-efficient 4D Gaussian representation. Furthermore, we introduce an entropy-constrained Gaussian deformation technique that uses a deformation field to expand the action range of each Gaussian and integrates an opacity-based entropy loss to limit the number of Gaussians, thus forcing our model to use as few Gaussians as possible to fit a dynamic scene well. With simple half-precision storage and zip compression, our framework achieves a storage reduction by approximately 190$\times$ and 125$\times$ on the Technicolor and Neural 3D Video datasets, respectively, compared to the original 4DGS. Meanwhile, it maintains comparable rendering speeds and scene representation quality, setting a new standard in the field. Code is available at https://github.com/Xinjie-Q/MEGA.
OccluGaussian: Occlusion-Aware Gaussian Splatting for Large Scene Reconstruction and Rendering
Huawei Noah’s Ark Lab（华为诺亚实验室）；The Chinese University of Hong Kong (Shenzhen)（香港中文大学（深圳））；Shenzhen Institutes of Advanced Technology（深圳先进技术研究院）；The University of Hong Kong（香港大学）；Huawei Embodied Intelligence Lab（华为具身智能实验室）
CSS: Overcoming Pose and Scene Challenges in Crowd-Sourced 3D Gaussian Splatting
Beijing University of Posts and Telecommunications（北京邮电大学）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；China Unicom Smart City Research Institute（中国联通智慧城市研究院）
MegaSynth: Scaling Up 3D Scene Reconstruction with Synthesized Data
The University of Texas at Austin（德克萨斯大学奥斯汀分校）；Adobe Research（奥多比研究院）；Stony Brook University（石溪大学）；Oregon State University（俄勒冈州立大学）；Cornell University（康奈尔大学）
提出用程序化生成的70万场景数据集MegaSynth训练三维重建模型，消除语义信息实现可扩展，联合训练或预训练提升重建质量1.2-1.8 dB PSNR，仅用合成数据即可媲美真实数据训练效果。
ObjectGS: Object-aware Scene Reconstruction and Scene Understanding via Gaussian Splatting
University of Science and Technology of China（中国科学技术大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；The Chinese University of Hong Kong（香港中文大学）；The University of Hong Kong（香港大学）
3DitScene: Editing Any Scene via Language-guided Disentangled Gaussian Splatting
The Chinese University of Hong Kong（香港中文大学）；Stanford University（斯坦福大学）；Snap Inc.（Snap公司）；University of California Los Angeles（加利福尼亚大学洛杉矶分校）；ByteDance（字节跳动）
SurgicalGS: Dynamic 3D Gaussian Splatting for Accurate Robotic-Assisted Surgical Scene Reconstruction
The Hamlyn Centre for Robotic Surgery, Imperial College London（伦敦帝国理工学院哈姆林 robotic外科中心）；Hawkes Institute, University College London（伦敦大学学院霍克斯研究所）；University of Liverpool（利物浦大学）
