arXiv 2502.167792025-03-04cs.CVcs.AI

预训练模型时代的无位姿稀疏视角房间布局重建

Unposed Sparse Views Room Layout Reconstruction in the Age of Pretrain Model

Hong Kong Center for Construction Robotics, The Hong Kong University of Science and Technology（香港科技大学香港建造机器人中心）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Astribot；Intellifusion Inc.（云天励飞公司）；MMLab, The Chinese University of Hong Kong（香港中文大学MMLab实验室）

Yaxuan Huang, Xili Dai, Jianan Wang, Xianbiao Qi, Yixing Yuan, Xiangyu Yue

备注：: Accepted by ICLR 2025. Github page:https://github.com/justacar/Plane-DUSt3R

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

提出Plane-DUSt3R，基于3D基础模型DUSt3R微调以估计结构平面，通过端到端单步方法从多视角图像实现房间布局重建，减少误差累积并在合成及野外数据上表现优异。

摘要：

由于多视图几何带来的复杂性，从多视角图像进行房间布局估计的研究尚不充分，这需要多步解决方案，如相机内外参估计、图像匹配和三角测量。然而，在3D重建中，近期3D基础模型（如DUSt3R）的进展已将范式从传统的多步运动恢复结构过程转变为端到端的单步方法。为此，我们引入了Plane-DUSt3R，一种利用3D基础模型DUSt3R的多视图房间布局估计新方法。Plane-DUSt3R结合了DUSt3R框架，并在房间布局数据集（Structure3D）上进行微调，修改了目标以估计结构平面。通过生成统一且简洁的结果，Plane-DUSt3R仅需单一后处理步骤和2D检测结果即可实现房间布局估计。与以往依赖单视角或全景图像的方法不同，Plane-DUSt3R扩展了设置以处理多视角图像。此外，它提供了一种简化的端到端解决方案，简化了流程并减少了误差累积。实验结果表明，Plane-DUSt3R不仅在合成数据集上优于最先进的方法，而且在具有不同图像风格（如卡通）的野外数据上也证明了其鲁棒性和有效性。我们的代码可在以下网址获取：https://github.com/justacar/Plane-DUSt3R

英文摘要：

Room layout estimation from multiple-perspective images is poorly investigated due to the complexities that emerge from multi-view geometry, which requires muti-step solutions such as camera intrinsic and extrinsic estimation, image matching, and triangulation. However, in 3D reconstruction, the advancement of recent 3D foundation models such as DUSt3R has shifted the paradigm from the traditional multi-step structure-from-motion process to an end-to-end single-step approach. To this end, we introduce Plane-DUSt3R, a novel method for multi-view room layout estimation leveraging the 3D foundation model DUSt3R. Plane-DUSt3R incorporates the DUSt3R framework and fine-tunes on a room layout dataset (Structure3D) with a modified objective to estimate structural planes. By generating uniform and parsimonious results, Plane-DUSt3R enables room layout estimation with only a single post-processing step and 2D detection results. Unlike previous methods that rely on single-perspective or panorama image, Plane-DUSt3R extends the setting to handle multiple-perspective images. Moreover, it offers a streamlined, end-to-end solution that simplifies the process and reduces error accumulation. Experimental results demonstrate that Plane-DUSt3R not only outperforms state-of-the-art methods on the synthetic dataset but also proves robust and effective on in the wild data with different image styles such as cartoon. Our code is available at: https://github.com/justacar/Plane-DUSt3R

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。