NOVA3R:非像素对齐的视觉变换器用于模态3D重建
NOVA3R: Non-pixel-aligned Visual Transformer for Amodal 3D Reconstruction
AI总结:
NOVA3R通过非像素对齐的视觉变换器实现更准确的3D重建,解决了传统像素对齐方法在处理可见和不可见点及重叠区域结构上的局限。
摘要:
我们提出了NOVA3R,一种有效的非像素对齐3D重建方法,从一组未定位图像中进行前馈处理。与将几何与每条光线预测绑定在一起的像素对齐方法不同,我们的方法学习了一个全局的、视图无关的场景表示,将重建与像素对齐解耦。这解决了像素对齐3D的两个关键限制:(1)它能够通过完整的场景表示恢复可见和不可见的点;(2)它在重叠区域产生物理上合理的几何结构,减少了重复结构。为此,我们引入了场景令牌机制,该机制在未定位图像之间聚合信息,并引入了基于扩散的3D解码器,以重建完整的、非像素对齐的点云。在场景级和物体级数据集上的广泛实验表明,NOVA3R在重建准确性和完整性方面优于最先进的方法。
英文摘要:
We present NOVA3R, an effective approach for non-pixel-aligned 3D reconstruction from a set of unposed images in a feed-forward manner. Unlike pixel-aligned methods that tie geometry to per-ray predictions, our formulation learns a global, view-agnostic scene representation that decouples reconstruction from pixel alignment. This addresses two key limitations in pixel-aligned 3D: (1) it recovers both visible and invisible points with a complete scene representation, and (2) it produces physically plausible geometry with fewer duplicated structures in overlapping regions. To achieve this, we introduce a scene-token mechanism that aggregates information across unposed images and a diffusion-based 3D decoder that reconstructs complete, non-pixel-aligned point clouds. Extensive experiments on both scene-level and object-level datasets demonstrate that NOVA3R outperforms state-of-the-art methods in terms of reconstruction accuracy and completeness.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。