ERF-GS：从不相交的事件-RGB视点重建快速运动
ERF-GS: Reconstructing Fast Motion from Disjoint Event-RGB Viewpoints
浏览论文内容
中文总结 AI 辅助
ERF-GS框架将事件信息融入高斯溅射流水线，脱离RGB输入实现基于事件的学习，在含模糊帧和不相交视点的数据集上，性能优于4DGS与E-D3DGS，可用于复杂自然视频的快速运动重建。
中文摘要 AI 辅助
神经辐射场（NeRF）和3D高斯溅射（3DGS）等深度学习驱动的表示方法，凭借更高的视觉精度和可扩展性，彻底改变了动态3D场景重建领域。然而，快速运动物体的重建仍是一项挑战；现有基于传统帧视频的方法在体育赛事、动物摄影等场景中往往表现不佳。我们提出一种事件-RGB融合高斯溅射（ERF-GS）框架，该框架将事件信息整合到高斯溅射流水线的优化和致密化阶段，利用了具有高帧率的新型事件传感器。与许多其他事件辅助场景重建方法不同，ERF-GS在真实模拟设置下开发，实现了脱离RGB输入的基于事件的学习。该设计使其应用范围超出简单的合成数据，扩展到布局复杂、帧率低且运动模糊严重的自然视频领域。实验表明，在包含模糊RGB帧和不相交RGB-事件视点的Neu3D及Nvidia数据集的不同变体上，ERF-GS的性能优于4DGS基线和同期的E-D3DGS。我们的代码可在该https URL获取。
英文摘要
Deep learning-driven representations such as neural radiance fields (NeRFs) and 3D Gaussian splatting (3DGS) have revolutionized the field of dynamic 3D scene reconstruction with improved visual precision and scalability. However, the reconstruction of fast-moving objects remains a challenge; existing methods based on conventional frame-based videos often struggle in scenarios such as sports events and animal videography. We propose an event-RGB fusion Gaussian splatting (ERF-GS) framework that integrates event information into both optimization and densification stages of the Gaussian splatting pipeline, taking advantage of novel event sensors with high frame-rate. Unlike many other event-assisted scene reconstruction methods, ERF-GS was developed using realistic simulation settings and realizes event-based learning detached from RGB inputs. This design enables its application beyond straightforward synthetic data into the realm of natural video with complex layout, low frame rates and severe motion blur. Our experiments show that ERF-GS outperforms both the 4DGS baseline and the concurrent E-D3DGS on different variants of the Neu3D and Nvidia datasets which include blurry RGB frames and disjoint RGB-event viewpoints. Our code is available at https://github.com/andrewbxy/ERF-GS.
发表机构
- The University of Hong Kong（香港大学）
- Zhejiang University（浙江大学）
机构由 AI 辅助整理，请以论文原文为准。
