arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2303.04928cs.RO
收藏

学习探索策略来解决现实世界的大理石问题

Learning Exploration Strategies to Solve Real-World Marble Runs

Carnegie Mellon University(卡内基梅隆大学)

Alisa Allaire, Christopher G. Atkeson

摘要:

涉及局部不稳定或不连续动力学(如分叉和碰撞)的任务在机器人技术中仍然具有挑战性,因为环境中的微小变化会对任务结果产生重大影响。对于这样的任务,学习鲁棒的确定性策略是困难的。我们专注于结构化探索与多个随机政策的基础上的混合专家模型(MoE)的政策表示,可以有效地适应。MoE策略由随机子策略和高级选择策略组成,随机子策略允许探索动作空间(或策略)的多个不同区域,高级选择策略引导探索最有希望的区域。我们开发了一个机器人系统,以评估我们的方法在现实世界中的物理问题解决域。在模拟中训练了MoE策略之后,现实世界中的在线学习在几十次尝试中就证明了有效的适应性,并且sim2real差距最小。我们的研究结果证实,代表多个策略,促进有效的适应新的环境和策略下学习不同的动态仍然可以提供有用的信息,在哪里寻找好的策略。

英文摘要:

Tasks involving locally unstable or discontinuous dynamics (such as bifurcations and collisions) remain challenging in robotics, because small variations in the environment can have a significant impact on task outcomes. For such tasks, learning a robust deterministic policy is difficult. We focus on structuring exploration with multiple stochastic policies based on a mixture of experts (MoE) policy representation that can be efficiently adapted. The MoE policy is composed of stochastic sub-policies that allow exploration of multiple distinct regions of the action space (or strategies) and a high-level selection policy to guide exploration towards the most promising regions. We develop a robot system to evaluate our approach in a real-world physical problem solving domain. After training the MoE policy in simulation, online learning in the real world demonstrates efficient adaptation within just a few dozen attempts, with a minimal sim2real gap. Our results confirm that representing multiple strategies promotes efficient adaptation in new environments and strategies learned under different dynamics can still provide useful information about where to look for good strategies.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑