任务不可知形态演化
Task-Agnostic Morphology Evolution
摘要：
深度强化学习主要关注学习行为，通常忽略了一个事实，即智能体的功能在很大程度上取决于形式。那么，在给定的环境中，应该如何找到适合解决任务的形态学呢？目前的方法，共同适应形态和行为使用一个特定的任务的奖励作为信号形态优化。然而，这通常需要昂贵的策略优化，并导致任务依赖的形态，而不是建立泛化。在这项工作中，我们提出了一种新的方法，任务不可知形态进化（TAME），以减轻这两个问题。在没有任何任务或奖励规范的情况下，TAME通过仅在代理群体上应用随机采样的动作原语来进化形态。这是通过使用信息理论目标来实现的，该目标通过代理在环境中达到不同状态的能力以及其行为的因果关系来有效地对代理进行排名。最后，我们的经验表明，在2D，3D和操作环境TAME可以演变的形态相匹配的多任务性能与任务监督算法的学习。我们的代码和视频可以在https://sites.google.com/view/task-agnostic-evolution上找到。
英文摘要：
Deep reinforcement learning primarily focuses on learning behavior, usually overlooking the fact that an agent's function is largely determined by form. So, how should one go about finding a morphology fit for solving tasks in a given environment? Current approaches that co-adapt morphology and behavior use a specific task's reward as a signal for morphology optimization. However, this often requires expensive policy optimization and results in task-dependent morphologies that are not built to generalize. In this work, we propose a new approach, Task-Agnostic Morphology Evolution (TAME), to alleviate both of these issues. Without any task or reward specification, TAME evolves morphologies by only applying randomly sampled action primitives on a population of agents. This is accomplished using an information-theoretic objective that efficiently ranks agents by their ability to reach diverse states in the environment and the causality of their actions. Finally, we empirically demonstrate that across 2D, 3D, and manipulation environments TAME can evolve morphologies that match the multi-task performance of those learned with task supervised algorithms. Our code and videos can be found at https://sites.google.com/view/task-agnostic-evolution.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
