arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1707.07907cs.AI
收藏

相互对齐迁移学习

Mutual Alignment Transfer Learning

Oxford Robotics Institute(牛津机器人研究所);University of Oxford(牛津大学);Berkeley AI Research (BAIR)(伯克利人工智能研究所);University of California Berkeley(加利福尼亚大学伯克利分校);OpenAI, San Francisco(OpenAI,旧金山)

Markus Wulfmeier, Ingmar Posner, Pieter Abbeel

摘要:

训练机器人在现实世界中操作是一项复杂、耗时且可能昂贵的任务。尽管强化学习在游戏和模拟中取得了巨大的成功,但在实际机器人应用中的研究还无法与类似的进展相匹配。虽然可以通过在模拟中训练策略来降低样本复杂性,但由于模型动态的不完美校准,这些策略在真实平台上的表现可能不佳。我们提出了一种方法-补充真正的机器人微调-进一步受益于并行访问模拟器在训练过程中,并减少对真正的机器人样本的要求。所开发的方法利用辅助奖励来指导探索的基础上,在模拟的代理,反之亦然的熟练程度为现实世界的代理。在这种情况下,我们经验证明,这两个代理的相互对齐提供了进一步的好处,因为模拟中的代理可以调整,以优化其行为的状态,通常由现实世界的代理访问。

英文摘要:

Training robots for operation in the real world is a complex, time consuming and potentially expensive task. Despite significant success of reinforcement learning in games and simulations, research in real robot applications has not been able to match similar progress. While sample complexity can be reduced by training policies in simulation, such policies can perform sub-optimally on the real platform given imperfect calibration of model dynamics. We present an approach -- supplemental to fine tuning on the real robot -- to further benefit from parallel access to a simulator during training and reduce sample requirements on the real robot. The developed approach harnesses auxiliary rewards to guide the exploration for the real world agent based on the proficiency of the agent in simulation and vice versa. In this context, we demonstrate empirically that the reciprocal alignment for both agents provides further benefit as the agent in simulation can adjust to optimize its behaviour for states commonly visited by the real-world agent.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑