arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2302.11070cs.AIcs.ROstat.ML
收藏

通过上下文调制实现通用形态控制

Universal Morphology Control via Contextual Modulation

University of Oxford(牛津大学)

Zheng Xiong, Jacob Beck, Shimon Whiteson

AI总结:

提出一种基于上下文调制的分层架构,通过hypernetworks生成形态相关参数及固定注意力机制调节肢体交互,有效解决了多机器人形态通用策略学习问题,显著提升了训练性能与零样本泛化能力。

摘要:

在不同机器人形态之间学习通用策略可以显著提高连续控制中的学习效率和泛化能力。然而,这提出了一个具有挑战性的多任务强化学习问题,因为不同机器人的最优策略可能差异很大,并且关键取决于形态。现有方法利用图神经网络或transformers来处理不同形态间的异构状态和动作空间,但很少关注机器人的控制策略对其形态上下文的依赖。在本文中,我们提出了一种分层架构,通过上下文调制来更好地建模这种依赖关系,该架构包含两个关键子模块:(1)我们不再在不同机器人之间强制执行硬参数共享,而是使用hypernetworks生成依赖于形态的控制参数;(2)我们提出了一种固定注意力机制,该机制仅依赖于形态来调制机器人中不同肢体之间的交互。实验结果表明,我们的方法不仅提高了在多种训练机器人上的学习性能,而且能以零样本方式更好地泛化到未见形态。

英文摘要:

Learning a universal policy across different robot morphologies can significantly improve learning efficiency and generalization in continuous control. However, it poses a challenging multi-task reinforcement learning problem, as the optimal policy may be quite different across robots and critically depend on the morphology. Existing methods utilize graph neural networks or transformers to handle heterogeneous state and action spaces across different morphologies, but pay little attention to the dependency of a robot's control policy on its morphology context. In this paper, we propose a hierarchical architecture to better model this dependency via contextual modulation, which includes two key submodules: (1) Instead of enforcing hard parameter sharing across robots, we use hypernetworks to generate morphology-dependent control parameters; (2) We propose a fixed attention mechanism that solely depends on the morphology to modulate the interactions between different limbs in a robot. Experimental results show that our method not only improves learning performance on a diverse set of training robots, but also generalizes better to unseen morphologies in a zero-shot fashion.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑