企业化逆向课程生成
Parallelized Reverse Curriculum Generation
摘要：
对于强化学习（RL），由于稀疏的奖励，智能体掌握需要特定一系列动作的任务是具有挑战性的。为了解决这个问题，反向课程生成（RCG）提供了一个反向扩展的方法，自动生成一个课程的代理学习。更具体地说，随着训练的进行，RCG将初始状态分布从目标的邻域调整到距离。然而，为每次迭代生成的初始状态分布可能是有偏的，从而使策略过拟合或减慢反向扩展速率。在为基于行动者-评论者（AC）的RL算法训练RCG时，AC对之间的紧密耦合可能导致这种较差的泛化和缓慢的收敛。因此，我们提出了一种并行化的方法，同时训练多个AC对，并定期交换他们的批评。实验表明，该方法可以提高RCG的性能和收敛性，它也可以应用到其他AC为基础的RL算法与适应的初始状态分布。
英文摘要：
For reinforcement learning (RL), it is challenging for an agent to master a task that requires a specific series of actions due to sparse rewards. To solve this problem, reverse curriculum generation (RCG) provides a reverse expansion approach that automatically generates a curriculum for the agent to learn. More specifically, RCG adapts the initial state distribution from the neighborhood of a goal to a distance as training proceeds. However, the initial state distribution generated for each iteration might be biased, thus making the policy overfit or slowing down the reverse expansion rate. While training RCG for actor-critic (AC) based RL algorithms, this poor generalization and slow convergence might be induced by the tight coupling between an AC pair. Therefore, we propose a parallelized approach that simultaneously trains multiple AC pairs and periodically exchanges their critics. We empirically demonstrate that this proposed approach can improve RCG in performance and convergence, and it can also be applied to other AC based RL algorithms with adapted initial state distribution.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
