WorldEval：世界模型作为真实世界机器人策略评估器
WorldEval: World Model as Real-World Robot Policies Evaluator
AI总结：
本文提出WorldEval，利用世界模型作为真实世界机器人策略的在线评估器，通过Policy2Vec将视频生成模型转为世界模拟器，实现策略排名与安全检测，并与真实环境性能强相关。
摘要：
机器人领域在开发通用机器人操作策略方面取得了显著进展。然而，在真实世界场景中评估这些策略仍然耗时且具有挑战性，尤其是当任务数量增加和环境条件变化时。在这项工作中，我们证明了世界模型可以作为真实世界机器人策略评估的可扩展、可复现且可靠的代理。一个关键挑战是从世界模型生成准确反映机器人动作的策略视频。我们观察到，直接输入机器人动作或使用高维编码方法往往无法生成遵循动作的视频。为了解决这个问题，我们提出了Policy2Vec，一种简单而有效的方法，将视频生成模型转变为遵循潜在动作生成机器人视频的世界模拟器。然后，我们引入了WorldEval，一个自动化流水线，旨在完全在线评估真实世界机器人策略。WorldEval有效地对多种机器人策略和单个策略内的各个检查点进行排名，并作为安全检测器，防止新开发的机器人模型产生危险动作。通过对真实世界环境中操作策略的全面配对评估，我们证明了WorldEval中的策略性能与真实世界场景之间存在强相关性。此外，我们的方法显著优于流行的真实到模拟（real-to-sim）方法。
英文摘要：
The field of robotics has made significant strides toward developing generalist robot manipulation policies. However, evaluating these policies in real-world scenarios remains time-consuming and challenging, particularly as the number of tasks scales and environmental conditions change. In this work, we demonstrate that world models can serve as a scalable, reproducible, and reliable proxy for real-world robot policy evaluation. A key challenge is generating accurate policy videos from world models that faithfully reflect the robot actions. We observe that directly inputting robot actions or using high-dimensional encoding methods often fails to generate action-following videos. To address this, we propose Policy2Vec, a simple yet effective approach to turn a video generation model into a world simulator that follows latent action to generate the robot video. We then introduce WorldEval, an automated pipeline designed to evaluate real-world robot policies entirely online. WorldEval effectively ranks various robot policies and individual checkpoints within a single policy, and functions as a safety detector to prevent dangerous actions by newly developed robot models. Through comprehensive paired evaluations of manipulation policies in real-world environments, we demonstrate a strong correlation between policy performance in WorldEval and real-world scenarios. Furthermore, our method significantly outperforms popular methods such as real-to-sim approach.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
