基于分段演示的子任务感知视觉奖励学习
Subtask-Aware Visual Reward Learning from Segmented Demonstrations
AI总结：
本文提出REDS框架，利用分段视频演示和最小监督学习密集奖励函数，通过对比学习对齐子任务，在复杂操作任务中显著优于基线并支持泛化。
摘要：
强化学习（RL）智能体已在各种机器人任务中展示了其潜力。然而，它们仍然严重依赖人工设计的奖励函数，这需要大量的试错和对目标行为信息的访问，而这些信息在现实世界环境中往往不可用。本文介绍了REDS：基于分段演示的奖励学习（REward learning from Demonstration with Segmentations），一种新颖的奖励学习框架，利用无动作视频和最小监督。具体而言，REDS采用来自不同来源的分段为子任务的视频演示，并将这些片段视为真实奖励。我们训练一个以视频片段及其对应子任务为条件的密集奖励函数，通过最小化等价策略不变比较距离来确保与真实奖励信号对齐。此外，我们采用对比学习目标来对齐视频表示与子任务，确保在线交互期间精确的子任务推断。我们的实验表明，在Meta-World的复杂机器人操作任务和更具挑战性的现实任务（如FurnitureBench中的家具组装）中，REDS在最小人工干预下显著优于基线方法。此外，REDS促进了对未见任务和机器人形态的泛化，突显了其在多样化环境中可扩展部署的潜力。
英文摘要：
Reinforcement Learning (RL) agents have demonstrated their potential across various robotic tasks. However, they still heavily rely on human-engineered reward functions, requiring extensive trial-and-error and access to target behavior information, often unavailable in real-world settings. This paper introduces REDS: REward learning from Demonstration with Segmentations, a novel reward learning framework that leverages action-free videos with minimal supervision. Specifically, REDS employs video demonstrations segmented into subtasks from diverse sources and treats these segments as ground-truth rewards. We train a dense reward function conditioned on video segments and their corresponding subtasks to ensure alignment with ground-truth reward signals by minimizing the Equivalent-Policy Invariant Comparison distance. Additionally, we employ contrastive learning objectives to align video representations with subtasks, ensuring precise subtask inference during online interactions. Our experiments show that REDS significantly outperforms baseline methods on complex robotic manipulation tasks in Meta-World and more challenging real-world tasks, such as furniture assembly in FurnitureBench, with minimal human intervention. Moreover, REDS facilitates generalization to unseen tasks and robot embodiments, highlighting its potential for scalable deployment in diverse environments.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
