Gamba：将高斯泼溅与Mamba结合用于单视图3D重建
Gamba: Marry Gaussian Splatting with Mamba for single view 3D reconstruction
AI总结：
Gamba是一种端到端单视图3D重建模型，结合Mamba骨干与高斯泼溅，通过径向掩码约束实现毫秒级重建，比优化方法快千倍。
摘要：
我们解决了从单张图像以毫秒级速度高效重建3D资产的挑战。现有的单图像3D重建方法主要基于分数蒸馏采样（SDS）与神经3D表示。尽管取得了有希望的结果，这些方法因耗时的优化和显著的内存消耗而面临实际限制。在本工作中，我们引入了Gamba，一个从单视图图像进行端到端3D重建的模型，强调两个主要见解：（1）高效骨干设计：引入基于Mamba的GambaFormer网络，将3D高斯泼溅（3DGS）重建建模为具有线性可扩展性的序列预测，从而容纳大量的高斯；（2）稳健的高斯约束：从多视图掩码推导径向掩码约束，以消除训练中3D点云预热监督的需要。我们在Objaverse上训练了Gamba，并在GSO数据集上将其与现有的基于优化和前馈的3D重建方法进行了评估，其中Gamba是唯一使用3DGS进行端到端训练的单视图重建模型。实验结果在定性和定量上都展示了其竞争力的生成能力，并突出了其显著的速度：Gamba在单个NVIDIA A100 GPU上于0.05秒内完成重建，比基于优化的方法快约1000倍。请参阅我们的项目页面：https://florinshen.github.io/gamba-project。
英文摘要：
We tackle the challenge of efficiently reconstructing a 3D asset from a single image at millisecond speed. Existing methods for single-image 3D reconstruction are primarily based on Score Distillation Sampling (SDS) with Neural 3D representations. Despite promising results, these approaches encounter practical limitations due to lengthy optimizations and significant memory consumption. In this work, we introduce Gamba, an end-to-end 3D reconstruction model from a single-view image, emphasizing two main insights: (1) Efficient Backbone Design: introducing a Mamba-based GambaFormer network to model 3D Gaussian Splatting (3DGS) reconstruction as sequential prediction with linear scalability of token length, thereby accommodating a substantial number of Gaussians; (2) Robust Gaussian Constraints: deriving radial mask constraints from multi-view masks to eliminate the need for warmup supervision of 3D point clouds in training. We trained Gamba on Objaverse and assessed it against existing optimization-based and feed-forward 3D reconstruction approaches on the GSO Dataset, among which Gamba is the only end-to-end trained single-view reconstruction model with 3DGS. Experimental results demonstrate its competitive generation capabilities both qualitatively and quantitatively and highlight its remarkable speed: Gamba completes reconstruction within 0.05 seconds on a single NVIDIA A100 GPU, which is about $1,000\times$ faster than optimization-based methods. Please see our project page at https://florinshen.github.io/gamba-project.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
