arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.07648cs.CV
收藏

基于几何感知位置编码与注意力机制的多视角大型重建模型

Multi-View Large Reconstruction Model via Geometry-Aware Positional Encoding and Attention

Hong Kong University of Science and Technology(香港科技大学);University of Hong Kong(香港大学)

Mengfei Li, Xiaoxiao Long, Yixun Liang, Weiyu Li, Yuan Liu, Peng Li, Wenhan Luo, Wenping Wang, Yike Guo

AI总结:

针对大型重建模型(LRM)扩展到多视角输入时存在的效率低、质量差、收敛慢问题,提出多视角大型重建模型(M-LRM),通过多视角一致交叉注意力机制和3D先验初始化三平面令牌,实现高保真3D重建并提升性能与收敛速度。

摘要:

尽管大型重建模型(LRM)近期取得了显著进展并展现出令人印象深刻的结果,但当将其输入从单张图像扩展到多张图像时,它表现出效率低下、几何与纹理质量欠佳以及收敛速度慢于预期的问题。这是因为LRM将3D重建表述为简单的图像到3D的翻译问题,忽略了输入图像之间强烈的3D一致性。在本文中,我们提出了一种多视角大型重建模型(M-LRM),旨在以3D感知的方式从多视角重建高质量3D形状。具体而言,我们引入了一种多视角一致的交叉注意力机制,使M-LRM能够从输入图像中准确查询信息。此外,我们利用输入多视角图像的3D先验来初始化三平面令牌。与先前方法相比,所提出的M-LRM能够生成高保真度的3D形状。实验研究表明,我们的模型实现了显著的性能提升和更快的训练收敛速度。项目页面:\u003chttps://murphylmf.github.io/M-LRM/\u003e。

英文摘要:

Despite recent advancements in the Large Reconstruction Model (LRM) demonstrating impressive results, when extending its input from single image to multiple images, it exhibits inefficiencies, subpar geometric and texture quality, as well as slower convergence speed than expected. It is attributed to that, LRM formulates 3D reconstruction as a naive images-to-3D translation problem, ignoring the strong 3D coherence among the input images. In this paper, we propose a Multi-view Large Reconstruction Model (M-LRM) designed to reconstruct high-quality 3D shapes from multi-views in a 3D-aware manner. Specifically, we introduce a multi-view consistent cross-attention scheme to enable M-LRM to accurately query information from the input images. Moreover, we employ the 3D priors of the input multi-view images to initialize the triplane tokens. Compared to previous methods, the proposed M-LRM can generate 3D shapes of high fidelity. Experimental studies demonstrate that our model achieves a significant performance gain and faster training convergence. Project page: \url{https://murphylmf.github.io/M-LRM/}.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑