ZeroMimic：从网络视频中蒸馏机器人操作技能
ZeroMimic: Distilling Robotic Manipulation Skills from Web Videos
AI总结：
提出 ZeroMimic 系统，通过蒸馏网络人类视频数据集生成可立即部署的图像目标条件化机器人操作技能策略，无需特定机器人示范，在多种真实与模拟环境中展现出色泛化能力。
摘要：
近期机器人操作领域的许多进展源于模仿学习，但这些方法主要依赖一种极难获取的示范数据：即在测试时训练策略必须应对的相同房间、相同物体下，于同一机器人上收集的数据。相比之下，大量展示自然场景下操作技能的预录人类视频数据集已经存在，其中蕴含对机器人有价值的信息。是否有可能从这类数据中蒸馏出实用的机器人技能策略库，而无需对特定机器人的示范或探索提出额外要求？我们提出了首个此类系统 ZeroMimic，它针对若干常见操作任务类别（打开、关闭、倾倒、拾取与放置、切割和搅拌）生成可立即部署的图像目标条件化技能策略，每种策略均能作用于不同物体并适应各种未见过的任务设置。ZeroMimic 经过精心设计，利用了人类视频语义与几何视觉理解的最新进展，结合现代抓取可供性检测器和模仿策略类别。在流行的 EpicKitchens 第一人称视角人类视频数据集上训练 ZeroMimic 后，我们在多种真实世界和模拟厨房环境中，使用两种不同的机器人本体评估了其开箱即用的性能，展示了其处理这些多样任务的出色能力。为实现 ZeroMimic 策略在其他任务设置和机器人上的即插即用复用，我们发布了技能策略的软件和检查点。
英文摘要：
Many recent advances in robotic manipulation have come through imitation learning, yet these rely largely on mimicking a particularly hard-to-acquire form of demonstrations: those collected on the same robot in the same room with the same objects as the trained policy must handle at test time. In contrast, large pre-recorded human video datasets demonstrating manipulation skills in-the-wild already exist, which contain valuable information for robots. Is it possible to distill a repository of useful robotic skill policies out of such data without any additional requirements on robot-specific demonstrations or exploration? We present the first such system ZeroMimic, that generates immediately deployable image goal-conditioned skill policies for several common categories of manipulation tasks (opening, closing, pouring, pick&place, cutting, and stirring) each capable of acting upon diverse objects and across diverse unseen task setups. ZeroMimic is carefully designed to exploit recent advances in semantic and geometric visual understanding of human videos, together with modern grasp affordance detectors and imitation policy classes. After training ZeroMimic on the popular EpicKitchens dataset of ego-centric human videos, we evaluate its out-of-the-box performance in varied real-world and simulated kitchen settings with two different robot embodiments, demonstrating its impressive abilities to handle these varied tasks. To enable plug-and-play reuse of ZeroMimic policies on other task setups and robots, we release software and policy checkpoints of our skill policies.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
