3D Gaussian Splatting(3DGS,三维高斯溅射)在静态场景重建方面已展现出显著潜力,近期进展也将其应用拓展至动态场景。但重建质量高度依赖高质量输入图像与精确相机位姿,这在真实场景中很难轻易实现。例如,使用手持单目相机拍摄动态场景时,相机与场景内物体通常会在单次曝光期间同时运动。这种组合运动常会导致图像模糊,而现有方法无法妥善处理该问题。为应对这些挑战,我们提出BARD-GS,这是一种可实现鲁棒动态场景重建的新方法,能有效处理模糊输入与不精确的相机位姿。该方法包含两大核心组件:1)相机运动去模糊;2)物体运动去模糊。通过将运动模糊显式分解为相机运动模糊与物体运动模糊并分别建模,我们在动态区域取得了显著提升的渲染效果。此外,我们还收集了一个真实世界动态场景运动模糊数据集用于评估所提方法。大量实验表明,BARD-GS能在真实条件下有效重建高质量动态场景,性能显著优于现有方法。
英文摘要:
3D Gaussian Splatting (3DGS) has shown remarkable potential for static scene reconstruction, and recent advancements have extended its application to dynamic scenes. However, the quality of reconstructions depends heavily on high-quality input images and precise camera poses, which are not that trivial to fulfill in real-world scenarios. Capturing dynamic scenes with handheld monocular cameras, for instance, typically involves simultaneous movement of both the camera and objects within a single exposure. This combined motion frequently results in image blur that existing methods cannot adequately handle. To address these challenges, we introduce BARD-GS, a novel approach for robust dynamic scene reconstruction that effectively handles blurry inputs and imprecise camera poses. Our method comprises two main components: 1) camera motion deblurring and 2) object motion deblurring. By explicitly decomposing motion blur into camera motion blur and object motion blur and modeling them separately, we achieve significantly improved rendering results in dynamic regions. In addition, we collect a real-world motion blur dataset of dynamic scenes to evaluate our approach. Extensive experiments demonstrate that BARD-GS effectively reconstructs high-quality dynamic scenes under realistic conditions, significantly outperforming existing methods.
DOF-GS:Adjustable Depth-of-Field 3D Gaussian Splatting for Post-Capture Refocusing, Defocus Rendering and Blur Removal
State Key Laboratory of General Artificial Intelligence, Peking University(通用人工智能国家重点实验室,北京大学);School of Intelligence Science and Technology, Peking University(智能科学与技术学院,北京大学);University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校)
MBA-SLAM: Motion Blur Aware Gaussian Splatting SLAM
College of Computer Science and Technology at Zhejiang University(浙江大学计算机科学与技术学院);School of Engineering at Westlake University(西湖大学工程学院);School of Engineering, Westlake University(西湖大学工程学院)
Event-guided 3D Gaussian Splatting for Dynamic Human and Scene Reconstruction
College of Optical Science and Engineering, Zhejiang University, China(浙江大学光学工程学院);School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore(南洋理工大学电子与电气工程学院);School of Artificial Intelligence and Robotics, Hunan University, China(湖南大学人工智能与机器人学院)
JADE-GS: Joint Allocation of Deblurring Evidence for Event-Assisted 3D Gaussian Splatting
School of Mechatronic Engineering and Automation, Shanghai University(上海大学机电工程与自动化学院);Shanghai Baoshan Shangda General Intelligent Robotics Research Institute(上海宝山尚大通用智能机器人研究院);School of Computer Science and Technology, Tongji University(同济大学计算机科学与技术学院)
Deblur-Avatar: Animatable Avatars from Motion-Blurred Monocular Videos
School of Artificial Intelligence and Automation, Huazhong University of Science and Technology(华中科技大学人工智能与自动化学院);S-Lab for Advanced Intelligence, Nanyang Technological University(南洋理工大学先进智能实验室);College of Computing and Data Science, Nanyang Technological University(南洋理工大学计算与数据科学学院);SenseTime Research(商汤科技研究院)
FreGS: 3D Gaussian Splatting with Progressive Frequency Regularization
Nanyang Technological University(南洋理工大学);Max Planck Institute for Informatics(马克斯·普朗克信息学研究所);Carnegie Mellon University(卡内基梅隆大学);MBZUAI(穆罕默德·本·扎耶德人工智能大学)
Analytic-Splatting: Anti-Aliased 3D Gaussian Splatting via Analytic Integration
South China University of Technology(华南理工大学);Tencent AI Lab(腾讯AI实验室);City University of Hong Kong(香港城市大学);School of Data Science, The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳)数据科学学院)
PEGS: Physics-Event Enhanced Large Spatiotemporal Motion Reconstruction via 3D Gaussian Splatting
School of Electronic Information, Wuhan University(武汉大学电子信息学院);School of Computer Science, Wuhan University(武汉大学计算机学院);College of Mechanical and Vehicle Engineering, Chongqing University(重庆大学机械与车辆工程学院);School of Artificial Intelligence, Wuhan University(武汉大学人工智能学院)
DeblurNVS: Geometric Latent Diffusion for Novel View Synthesis from Sparse Motion-Blurred Images
School of AI for Science, Peking University Shenzhen Graduate School(人工智能科学学院,北京大学深圳研究生院);School of Electronic and Computer Engineering, Peking University Shenzhen Graduate School(电子与计算机工程学院,北京大学深圳研究生院)