Geo4D：利用视频生成器进行几何四维场景重建
Geo4D: Leveraging Video Generators for Geometric 4D Scene Reconstruction
AI总结：
Geo4D利用预训练视频扩散模型的动态先验，仅用合成数据训练即可零样本泛化，通过融合点图、视差图和射线图并采用滑动窗口推理，显著提升动态场景单目四维重建精度。
摘要：
我们提出Geo4D，一种将视频扩散模型重新用于动态场景单目三维重建的方法。通过利用大规模预训练视频模型捕获的强动态先验，Geo4D可以仅使用合成数据进行训练，同时以零样本方式良好地泛化到真实数据。Geo4D预测几种互补的几何模态，即点图、视差图和射线图。我们提出一种新的多模态对齐算法来对齐和融合这些模态，并在推理时采用滑动窗口方法，从而实现对长视频的鲁棒且精确的四维重建。在多个基准上的大量实验表明，Geo4D显著超越了最先进的视频深度估计方法。
英文摘要：
We introduce Geo4D, a method to repurpose video diffusion models for monocular 3D reconstruction of dynamic scenes. By leveraging the strong dynamic priors captured by large-scale pre-trained video models, Geo4D can be trained using only synthetic data while generalizing well to real data in a zero-shot manner. Geo4D predicts several complementary geometric modalities, namely point, disparity, and ray maps. We propose a new multi-modal alignment algorithm to align and fuse these modalities, as well as a sliding window approach at inference time, thus enabling robust and accurate 4D reconstruction of long videos. Extensive experiments across multiple benchmarks show that Geo4D significantly surpasses state-of-the-art video depth estimation methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
