arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2501.12387cs.CV
收藏

连续3D感知模型与持久状态

Continuous 3D Perception Model with Persistent State

University of California, Berkeley(加州大学伯克利分校);Google DeepMind(谷歌DeepMind)

Qianqian Wang, Yifei Zhang, Aleksander Holynski, Alexei A. Efros, Angjoo Kanazawa

AI总结:

提出统一框架CUT3R,利用持续更新的状态递归模型在线生成度量尺度点图,实现3D重建与未观测区域推断,在多种3D/4D任务上达到先进水平。

摘要:

我们提出了一个能够解决广泛3D任务的统一框架。我们的方法采用一个具有状态循环的递归模型,该模型随着每次新观测持续更新其状态表示。给定图像流,这种不断演变的状态可用于以在线方式为每个新输入生成度量尺度的点图(逐像素3D点)。这些点图位于一个公共坐标系中,并可以累积成一个连贯、密集的场景重建,随着新图像的到达而更新。我们提出的模型名为CUT3R(用于3D重建的连续更新Transformer),它捕获了真实世界场景的丰富先验:不仅可以从图像观测中预测准确的点图,还可以通过探测虚拟的、未观测的视图来推断场景中未观测到的区域。我们的方法简单而高度灵活,自然接受不同长度的图像,这些图像可以是视频流或无序的照片集合,包含静态和动态内容。我们在各种3D/4D任务上评估了我们的方法,并在每项任务中展示了具有竞争力或最先进的性能。项目页面:https://cut3r.github.io/

英文摘要:

We present a unified framework capable of solving a broad range of 3D tasks. Our approach features a stateful recurrent model that continuously updates its state representation with each new observation. Given a stream of images, this evolving state can be used to generate metric-scale pointmaps (per-pixel 3D points) for each new input in an online fashion. These pointmaps reside within a common coordinate system, and can be accumulated into a coherent, dense scene reconstruction that updates as new images arrive. Our model, called CUT3R (Continuous Updating Transformer for 3D Reconstruction), captures rich priors of real-world scenes: not only can it predict accurate pointmaps from image observations, but it can also infer unseen regions of the scene by probing at virtual, unobserved views. Our method is simple yet highly flexible, naturally accepting varying lengths of images that may be either video streams or unordered photo collections, containing both static and dynamic content. We evaluate our method on various 3D/4D tasks and demonstrate competitive or state-of-the-art performance in each. Project Page: https://cut3r.github.io/

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑