arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2511.16624cs.CVcs.AI
收藏

SAM 3D: 将图像中的任何内容3D化

SAM 3D: 3Dfy Anything in Images

Meta Superintelligence Labs(Meta超智能实验室)
SAM 3D Team, Xingyu Chen, Fu-Jen Chu, Pierre Gleize, Kevin J Liang, Alexander Sax, Hao Tang, Weiyao Wang, Michelle Guo, Thibaut Hardin, Xiang Li, Aohan Lin, Jia… 展开作者

SAM 3D Team, Xingyu Chen, Fu-Jen Chu, Pierre Gleize, Kevin J Liang, Alexander Sax, Hao Tang, Weiyao Wang, Michelle Guo, Thibaut Hardin, Xiang Li, Aohan Lin, Jiawei Liu, Ziqi Ma, Anushka Sagar, Bowen Song, Xiaodong Wang, Jianing Yang, Bowen Zhang, Piotr Dollár, Georgia Gkioxari, Matt Feiszli, Jitendra Malik

AI总结:

提出SAM 3D生成模型,从单张图像重建3D物体的几何、纹理和布局,通过人机协同标注和分阶段训练突破数据瓶颈,在真实场景中取得显著优势。

摘要:

我们提出SAM 3D,一种用于视觉引导的3D物体重建的生成模型,能够从单张图像预测几何、纹理和布局。SAM 3D在自然图像中表现出色,这些图像中遮挡和场景杂乱很常见,且来自上下文的视觉识别线索起着更重要的作用。我们通过一个人工和模型在环的流水线来标注物体形状、纹理和姿态,以前所未有的规模提供视觉引导的3D重建数据。我们在一个现代的、多阶段的训练框架中从这些数据中学习,该框架结合了合成预训练和真实世界对齐,打破了3D“数据壁垒”。与近期工作相比,我们获得了显著提升,在真实世界物体和场景的人类偏好测试中至少达到5:1的胜率。我们将发布我们的代码和模型权重、一个在线演示以及一个新的用于野外3D物体重建的具有挑战性的基准测试。

英文摘要:

We present SAM 3D, a generative model for visually grounded 3D object reconstruction, predicting geometry, texture, and layout from a single image. SAM 3D excels in natural images, where occlusion and scene clutter are common and visual recognition cues from context play a larger role. We achieve this with a human- and model-in-the-loop pipeline for annotating object shape, texture, and pose, providing visually grounded 3D reconstruction data at unprecedented scale. We learn from this data in a modern, multi-stage training framework that combines synthetic pretraining with real-world alignment, breaking the 3D "data barrier". We obtain significant gains over recent work, with at least a 5:1 win rate in human preference tests on real-world objects and scenes. We will release our code and model weights, an online demo, and a new challenging benchmark for in-the-wild 3D object reconstruction.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑