DrivingForward：基于灵活环视输入的前馈3D高斯泼溅驾驶场景重建
DrivingForward: Feed-forward 3D Gaussian Splatting for Driving Scene Reconstruction from Flexible Surround-view Input
- Shanghai Jiao Tong University（上海交通大学）
- East China Normal University（华东师范大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
提出DrivingForward模型，通过联合训练pose、depth和Gaussian网络，在无深度真值与相机外参下实现灵活环视输入的前馈3D驾驶场景实时重建，在nuScenes数据集上优于现有方法。
AI中文摘要：
我们提出了DrivingForward，一种前馈Gaussian Splatting模型，能从灵活的环视输入中重建驾驶场景。车载摄像头的驾驶场景图像通常较为稀疏，重叠有限，且车辆运动进一步增加了获取相机外参的难度。为解决这些挑战并实现实时重建，我们联合训练了pose网络、depth网络和Gaussian网络，以预测表示驾驶场景的Gaussian primitives。pose网络和depth网络以自监督方式确定Gaussian primitives的位置，在训练期间不使用深度真值和相机外参。Gaussian网络从每张输入图像中独立预测primitive参数，包括协方差、不透明度和球谐系数。在推理阶段，我们的模型能从灵活的多帧环视输入中实现前馈重建。在nuScenes数据集上的实验表明，在重建方面，我们的模型优于现有的最先进前馈和场景优化重建方法。
英文摘要：
We propose DrivingForward, a feed-forward Gaussian Splatting model that reconstructs driving scenes from flexible surround-view input. Driving scene images from vehicle-mounted cameras are typically sparse, with limited overlap, and the movement of the vehicle further complicates the acquisition of camera extrinsics. To tackle these challenges and achieve real-time reconstruction, we jointly train a pose network, a depth network, and a Gaussian network to predict the Gaussian primitives that represent the driving scenes. The pose network and depth network determine the position of the Gaussian primitives in a self-supervised manner, without using depth ground truth and camera extrinsics during training. The Gaussian network independently predicts primitive parameters from each input image, including covariance, opacity, and spherical harmonics coefficients. At the inference stage, our model can achieve feed-forward reconstruction from flexible multi-frame surround-view input. Experiments on the nuScenes dataset show that our model outperforms existing state-of-the-art feed-forward and scene-optimized reconstruction methods in terms of reconstruction.
