arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1906.08857cs.NEcs.AI
收藏

循环和离散世界模型的深层神经进化

Deep Neuroevolution of Recurrent and Discrete World Models

Uber AI(优步人工智能实验室)

Sebastian Risi, Kenneth O. Stanley

摘要:

受我们自己的人类认知系统启发的神经架构,例如最近引入的世界模型,已被证明在各种不同的领域优于传统的深度强化学习(RL)方法。与大多数RL实验中采用的相对简单的架构不同,世界模型依赖于多个不同的神经组件,这些组件负责视觉信息处理、记忆和决策。然而,到目前为止,这些模型的组件必须单独训练,并通过各种专门的训练方法。本文展示了一个令人惊讶的发现,即具有相同精确部件的模型可以通过遗传算法(GA)进行端到端的有效训练,通过解决具有挑战性的赛车任务,达到与原始世界模型相当的性能。对进化的视觉和记忆系统的分析表明,它们包括与通过梯度下降训练的系统类似的有效表示。此外,与处理离散变量的梯度下降方法相比,遗传算法也直接处理此类表示,为潜在空间中的经典规划提供了机会。本文增加了额外的证据,证明深度神经进化对于复杂异构架构中需要复杂编排多个组件的任务的有效性。

英文摘要:

Neural architectures inspired by our own human cognitive system, such as the recently introduced world models, have been shown to outperform traditional deep reinforcement learning (RL) methods in a variety of different domains. Instead of the relatively simple architectures employed in most RL experiments, world models rely on multiple different neural components that are responsible for visual information processing, memory, and decision-making. However, so far the components of these models have to be trained separately and through a variety of specialized training methods. This paper demonstrates the surprising finding that models with the same precise parts can be instead efficiently trained end-to-end through a genetic algorithm (GA), reaching a comparable performance to the original world model by solving a challenging car racing task. An analysis of the evolved visual and memory system indicates that they include a similar effective representation to the system trained through gradient descent. Additionally, in contrast to gradient descent methods that struggle with discrete variables, GAs also work directly with such representations, opening up opportunities for classical planning in latent space. This paper adds additional evidence on the effectiveness of deep neuroevolution for tasks that require the intricate orchestration of multiple components in complex heterogeneous architectures.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑