在Veo世界模拟器中评估Gemini机器人策略
Evaluating Gemini Robotics Policies in a Veo World Simulator
AI总结：
本研究提出基于Veo视频模型的生成式评估系统，用于在Veo世界模拟器中全面评估Gemini机器人策略的性能，包括名义性能、分布外泛化及安全约束测试。
摘要：
生成式世界模型在模拟与视觉-运动策略在不同环境中的交互方面具有显著潜力。前沿视频模型可以以可扩展和通用的方式生成逼真的观察和环境交互。然而，视频模型在机器人学中的应用主要局限于分布内评估，即与策略训练或微调基础视频模型所用的场景相似的场景。在本报告中，我们证明视频模型可以用于机器人学中政策评估的整个谱系的使用案例：从评估名义性能到分布外（OOD）泛化，以及探测物理和语义安全。我们介绍了一个基于前沿视频基础模型（Veo）的生成式评估系统。该系统优化以支持机器人动作条件和多视图一致性，同时整合生成图像编辑和多视图完成，以合成沿多个泛化轴的现实世界场景的真实变体。我们证明该系统保留了视频模型的基础能力，以准确模拟被编辑以包含新交互对象、新视觉背景和新干扰对象的场景。这种保真度使能够准确预测不同策略在名义和OOD条件下的相对性能，确定不同泛化轴对策略性能的相对影响，并对策略进行红队行动以暴露违反物理或语义安全约束的行为。我们通过1600多个现实世界评估验证了这些能力，评估了八个Gemini Robotics策略检查点和五个双臂操作器的任务。
英文摘要：
Generative world models hold significant potential for simulating interactions with visuomotor policies in varied environments. Frontier video models can enable generation of realistic observations and environment interactions in a scalable and general manner. However, the use of video models in robotics has been limited primarily to in-distribution evaluations, i.e., scenarios that are similar to ones used to train the policy or fine-tune the base video model. In this report, we demonstrate that video models can be used for the entire spectrum of policy evaluation use cases in robotics: from assessing nominal performance to out-of-distribution (OOD) generalization, and probing physical and semantic safety. We introduce a generative evaluation system built upon a frontier video foundation model (Veo). The system is optimized to support robot action conditioning and multi-view consistency, while integrating generative image-editing and multi-view completion to synthesize realistic variations of real-world scenes along multiple axes of generalization. We demonstrate that the system preserves the base capabilities of the video model to enable accurate simulation of scenes that have been edited to include novel interaction objects, novel visual backgrounds, and novel distractor objects. This fidelity enables accurately predicting the relative performance of different policies in both nominal and OOD conditions, determining the relative impact of different axes of generalization on policy performance, and performing red teaming of policies to expose behaviors that violate physical or semantic safety constraints. We validate these capabilities through 1600+ real-world evaluations of eight Gemini Robotics policy checkpoints and five tasks for a bimanual manipulator.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
