进化选择性模仿：无演示者模仿学习的可解释主体
Evolutionary Selective Imitation: Interpretable Agents by Imitation Learning Without a Demonstrator
摘要：
我们提出了一种通过进化策略（ES）训练代理的新方法，在该方法中，我们迭代地改进一组要模仿的样本：从一个随机集开始，在每次迭代中，我们用迄今为止发现的最佳轨迹的样本替换样本的子集。该集合的评估过程是通过监督学习来训练随机初始化的神经网络（NN）以模仿该集合，然后针对环境执行所获得的策略。因此，我们的方法是一个ES的基础上的适应度函数，表示模仿不断发展的数据子集的有效性。这与其他ES技术直接覆盖策略的权重形成对比。通过观察代理选择用于学习的样本，可以比NN学习更明确地解释和评估代理的进化策略。在我们的实验中，我们训练了一个智能体来解决OpenAI Gym环境Bipedalwalker-v3，方法是用一个只有几千个参数的NN来模拟一个只有25个样本的进化选择集。我们进一步测试我们的方法上的Procgen游戏掠夺，并在这里显示，以及所提出的方法是一个可解释的，小，强大的和有效的替代其他ES或策略梯度方法。
英文摘要：
We propose a new method for training an agent via an evolutionary strategy (ES), in which we iteratively improve a set of samples to imitate: Starting with a random set, in every iteration we replace a subset of the samples with samples from the best trajectories discovered so far. The evaluation procedure for this set is to train, via supervised learning, a randomly initialised neural network (NN) to imitate the set and then execute the acquired policy against the environment. Our method is thus an ES based on a fitness function that expresses the effectiveness of imitating an evolving data subset. This is in contrast to other ES techniques that iterate over the weights of the policy directly. By observing the samples that the agent selects for learning, it is possible to interpret and evaluate the evolving strategy of the agent more explicitly than in NN learning. In our experiments, we trained an agent to solve the OpenAI Gym environment Bipedalwalker-v3 by imitating an evolutionarily selected set of only 25 samples with a NN with only a few thousand parameters. We further test our method on the Procgen game Plunder and show here as well that the proposed method is an interpretable, small, robust and effective alternative to other ES or policy gradient methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
