用于多任务块推的单扩散策略控制器，具有零样本模拟到现实转移
A Single Diffusion-Policy Controller for Multi-Task Block Pushing with Zero-Shot Sim-to-Real Transfer
浏览论文内容
中文总结 AI 辅助
研究旨在用强化学习从零训练单扩散策略用于多任务块推，提出含简单策略损失函数的框架，结合反向课程生成等应对探索挑战，评估其在不同条件下零样本从模拟到现实的转移能力，证明该流程有效。
中文摘要 AI 辅助
扩散策略在通过行为克隆为机器人表示和学习复杂动作方面展现出了有前景的实证性能。本文中，我们探索使用强化学习从零开始训练扩散策略用于多任务机器人操纵。具体而言，我们旨在训练一个针对多种形状块推任务的单扩散策略。所提出的框架具有一个简单的策略损失函数，它是基于行为克隆的扩散策略训练中使用的重新加权证据下界，并且能无缝用作强化学习算法中的策略学习模块。为应对因缺乏示范而产生的探索挑战，我们纳入了反向课程生成和以目标为中心的表示。结合扩散策略的表现力，我们的设计支持在稀疏奖励模拟设置中学习多任务块推策略。我们进一步评估训练好的扩散策略在包括目标位置、块形状、块重量和表面摩擦等不同环境条件下能否零样本转移到现实世界任务，结果表明该流程在测试的变化情况下能转移到我们的现实世界块推设置中。
英文摘要
Diffusion policies have shown promising empirical performance in representing and learning complex maneuvers for robots using behavior cloning (BC). In this paper, we explore training diffusion policies from scratch using reinforcement learning (RL) for multi-task robotic manipulation. Specifically, we aim to train a single diffusion policy for block-pushing tasks with multiple shapes. The proposed framework features a simple policy loss function, which is a reweighted evidence lower bound used in BC-based diffusion policy training and can seamlessly serve as the policy learning module in RL algorithms. To address the exploration challenges arising from the absence of demonstrations, we incorporate reverse curriculum generation and objective-centric representations. Combined with the expressiveness of diffusion policies, our design supports learning of multi-task block-pushing policies in our sparse-reward simulation setting. We further evaluate whether the trained diffusion policy transfers in zero-shot to real-world tasks under varying environmental conditions including goal positions, block shapes, block weights and surface friction, providing evidence that this pipeline can transfer to our real-world block-pushing setup under the tested variations.
发表机构
- Harvard University（哈佛大学）
- Georgia Institute of Technology（佐治亚理工学院）
机构由 AI 辅助整理，请以论文原文为准。
