从单张图像到运动策略：基于视频生成环境表示
From Single Images to Motion Policies via Video-Generation Environment Representations
AI总结：
提出VGER框架，利用视频生成模型从单张RGB图像构建环境表示，并训练符合几何结构的运动生成模型，实现无碰撞运动。
摘要：
自主机器人通常需要构建其周围环境的表示，并使其运动适应环境的几何结构。在此，我们解决从单张输入RGB图像构建与环境一致的、用于无碰撞运动生成的策略模型的问题。从单张图像提取3D结构通常涉及单目深度估计。深度估计的发展催生了诸如DepthAnything等大型预训练模型。然而，由于产生的视锥形误差，将这些模型的输出用于下游运动生成具有挑战性。相反，我们提出一个名为视频生成环境表示（VGER）的框架，该框架利用大规模视频生成模型的进展，生成以输入图像为条件的移动相机视频。该视频的帧构成多视图数据集，随后输入到预训练的3D基础模型以生成密集点云。然后，我们引入一种多尺度噪声方法来训练环境结构的隐式表示，并构建符合该表示几何结构的运动生成模型。我们在多样化的室内和室外环境中广泛评估VGER。我们证明了其能够从单张RGB输入图像生成考虑场景捕获几何结构的平滑运动。
英文摘要：
Autonomous robots typically need to construct representations of their surroundings and adapt their motions to the geometry of their environment. Here, we tackle the problem of constructing a policy model for collision-free motion generation, consistent with the environment, from a single input RGB image. Extracting 3D structures from a single image often involves monocular depth estimation. Developments in depth estimation have given rise to large pre-trained models such as DepthAnything. However, using outputs of these models for downstream motion generation is challenging due to frustum-shaped errors that arise. Instead, we propose a framework known as Video-Generation Environment Representation (VGER), which leverages the advances of large-scale video generation models to generate a moving camera video conditioned on the input image. Frames of this video, which form a multiview dataset, are then input into a pre-trained 3D foundation model to produce a dense point cloud. We then introduce a multi-scale noise approach to train an implicit representation of the environment structure and build a motion generation model that complies with the geometry of the representation. We extensively evaluate VGER over a diverse set of indoor and outdoor environments. We demonstrate its ability to produce smooth motions that account for the captured geometry of a scene, all from a single RGB input image.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
