arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.03594cs.CV
收藏

UniPlane:从有姿态的单目视频中统一进行平面检测与重建

UniPlane: Unified Plane Detection and Reconstruction from Posed Monocular Videos

University of Southern California(南加州大学);Meta Reality Labs(元宇宙现实实验室)

Yuzhong Huang, Chen Liu, Ji Hou, Ke Huo, Shiyu Dong, Fred Morstatter

AI总结:

UniPlane提出在单一Transformer网络中统一平面检测与重建,通过三维特征体和平面嵌入的点积直接重建平面,在真实数据集上以+4.6 F-score超越现有方法。

摘要:

我们提出UniPlane,一种从有姿态的单目视频中统一进行平面检测与重建的新方法。与现有方法从局部观测中检测平面并在视频中跨帧关联以完成最终重建不同,UniPlane在单一网络中将检测与重建任务统一起来,使我们能够直接优化最终重建质量并充分利用时间信息。具体来说,我们构建了一个基于Transformers的深度神经网络,该网络联合构建环境的三维特征体,并估计一组每平面嵌入作为查询。UniPlane通过对体素嵌入与平面嵌入进行点积运算并进行二值阈值化,直接重建三维平面。在真实世界数据集上的大量实验表明,UniPlane在平面检测和重建任务上均优于最先进方法,在几何F-score上取得+4.6的提升,并在其他几何和分割指标上取得一致改进。

英文摘要:

We present UniPlane, a novel method that unifies plane detection and reconstruction from posed monocular videos. Unlike existing methods that detect planes from local observations and associate them across the video for the final reconstruction, UniPlane unifies both the detection and the reconstruction tasks in a single network, which allows us to directly optimize final reconstruction quality and fully leverage temporal information. Specifically, we build a Transformers-based deep neural network that jointly constructs a 3D feature volume for the environment and estimates a set of per-plane embeddings as queries. UniPlane directly reconstructs the 3D planes by taking dot products between voxel embeddings and the plane embeddings followed by binary thresholding. Extensive experiments on real-world datasets demonstrate that UniPlane outperforms state-of-the-art methods in both plane detection and reconstruction tasks, achieving +4.6 in F-score in geometry as well as consistent improvements in other geometry and segmentation metrics.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑