arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2412.04827cs.CVcs.GR
收藏

PanoDreamer: 基于优化的单张图像到360度3D场景

PanoDreamer: Optimization-Based Single Image to 360 3D Scene With Diffusion

Texas A\&M University(德克萨斯A&M大学);Max Planck Institute for Informatics(马克斯·普朗克信息研究所);Leia Inc.(Leia公司)

Avinash Paliwal, Xilong Zhou, Andrii Tsarov, Nima Khademi Kalantari

AI总结:

PanoDreamer通过优化方法实现单图像到360度3D场景的重建,提出两种优化任务和交替最小化策略以提升一致性和质量。

摘要:

在本文中,我们提出PanoDreamer,一种从单张输入图像生成一致的360度3D场景的新方法。与现有方法生成场景顺序不同,我们将问题框架为单图像全景和深度估计。一旦获得一致的全景图像及其对应的深度,可以通过补全小的遮挡区域并将它们投影到3D空间中来重建场景。我们的关键贡献是将单图像全景和深度估计作为两个优化任务,并引入交替最小化策略来有效解决这些目标。我们证明了我们的方法在单图像360度3D场景重建方面在一致性和整体质量上优于现有技术。

英文摘要:

In this paper, we present PanoDreamer, a novel method for producing a coherent 360° 3D scene from a single input image. Unlike existing methods that generate the scene sequentially, we frame the problem as single-image panorama and depth estimation. Once the coherent panoramic image and its corresponding depth are obtained, the scene can be reconstructed by inpainting the small occluded regions and projecting them into 3D space. Our key contribution is formulating single-image panorama and depth estimation as two optimization tasks and introducing alternating minimization strategies to effectively solve their objectives. We demonstrate that our approach outperforms existing techniques in single-image 360° 3D scene reconstruction in terms of consistency and overall quality.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑