RoboVIP: 多视角视频生成与视觉身份提示增强机器人操作
RoboVIP: Multi-View Video Generation with Visual Identity Prompting Augments Robot Manipulation
AI总结：
RoboVIP通过引入视觉身份提示和可扩展的视觉身份池，提升机器人操控数据的质量与多样性，从而增强视觉语言-动作和视觉-运动政策模型的性能。
摘要：
操控数据的多样性、数量和质量对于训练有效的机器人策略至关重要。然而，由于硬件和物理设置的限制，收集大规模真实世界操控数据仍然难以在多样化环境中扩展。最近的研究使用文本提示条件图像扩散模型来通过改变视觉观察中的背景和桌面上的物体来增强操控数据。然而，这些方法往往忽略了最先进策略模型所需多视角和时间一致的观察的实用需求。此外，仅靠文本提示无法可靠地指定场景设置。为了给扩散模型提供明确的视觉指导，我们引入了视觉身份提示，它通过提供示例图像作为条件输入来指导所需场景设置的生成。为此，我们还构建了一个可扩展的流程，从大规模机器人数据集中整理出视觉身份池。使用我们增强的操控数据来训练下游的视觉语言-动作和视觉-运动政策模型，在仿真和真实机器人设置中都取得了持续的性能提升。
英文摘要：
The diversity, quantity, and quality of manipulation data are critical for training effective robot policies. However, due to hardware and physical setup constraints, collecting large-scale real-world manipulation data remains difficult to scale across diverse environments. Recent work uses text-prompt conditioned image diffusion models to augment manipulation data by altering the backgrounds and tabletop objects in the visual observations. However, these approaches often overlook the practical need for multi-view and temporally coherent observations required by state-of-the-art policy models. Further, text prompts alone cannot reliably specify the scene setup. To provide the diffusion model with explicit visual guidance, we introduce visual identity prompting, which supplies exemplar images as conditioning inputs to guide the generation of the desired scene setup. To this end, we also build a scalable pipeline to curate a visual identity pool from large robotics datasets. Using our augmented manipulation data to train downstream vision-language-action and visuomotor policy models yields consistent performance gains in both simulation and real-robot settings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
