当机器人比人类更强大时：从受限演示中学习
When a Robot is More Capable than a Human: Learning from Constrained Demonstrators
AI总结：
本文探讨机器人能否超越受限专家的演示学习，通过探索更高效轨迹和自定义奖励信号，提升学习效率和任务完成速度。
摘要：
从演示中学习使专家能通过接口如肢体教学、手柄控制和仿真到现实转移来教机器人复杂任务。然而，这些接口常因间接控制、设置限制和硬件安全而限制专家演示最优行为的能力。例如，手柄只能使机械臂在2D平面移动，尽管机器人在更高维空间中操作。因此，受限专家收集的演示导致学习策略性能低下。本文通过允许智能体超越直接模仿专家行为并探索更短高效的轨迹，使用演示推断仅状态奖励信号以衡量任务进度，并通过时间插值自标定未知状态的奖励。该方法在样本效率和任务完成时间上优于常规模仿学习。在真实WidowX机械臂上，其在12秒内完成任务，比行为克隆快10倍，如https://sites.google.com/view/constrainedexpert上的真实机器人视频所示。
英文摘要：
Learning from demonstrations enables experts to teach robots complex tasks using interfaces such as kinesthetic teaching, joystick control, and sim-to-real transfer. However, these interfaces often constrain the expert's ability to demonstrate optimal behavior due to indirect control, setup restrictions, and hardware safety. For example, a joystick can move a robotic arm only in a 2D plane, even though the robot operates in a higher-dimensional space. As a result, the demonstrations collected by constrained experts lead to suboptimal performance of the learned policies. This raises a key question: Can a robot learn a better policy than the one demonstrated by a constrained expert? We address this by allowing the agent to go beyond direct imitation of expert actions and explore shorter and more efficient trajectories. We use the demonstrations to infer a state-only reward signal that measures task progress, and self-label reward for unknown states using temporal interpolation. Our approach outperforms common imitation learning in both sample efficiency and task completion time. On a real WidowX robotic arm, it completes the task in 12 seconds, 10x faster than behavioral cloning, as shown in real-robot videos on https://sites.google.com/view/constrainedexpert .
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
