加速质量-通过大规模多元化实现多样性
Accelerated Quality-Diversity through Massive Parallelism
摘要：
质量多样性（QD）优化算法是一种众所周知的方法，以产生大量的多样化和高质量的解决方案。然而，量子点算法是一种基于种群的方法，它起源于演化计算，具有数据效率低、计算量大等缺点。这使得QD算法在解决方案评估计算成本高的应用中使用时变慢。加速QD算法的一种常见方法是并行评估解决方案，例如通过使用机器人中的物理模拟器。然而，这种方法仅限于几十个并行评估，因为大多数物理模拟器只能用更多数量的CPU来并行化。随着在加速器上运行的模拟器的最新进展，现在可以在单个GPU/TPU上并行执行数千次评估。在本文中，我们提出了QDax，一个加速实现的MAP-Elites，它利用加速器上的大规模并行性，使QD算法更容易访问。我们表明，QD算法是理想的候选人，以利用硬件加速的进展。我们证明，QD算法可以扩展与大规模的并行运行在交互式的时间尺度上的性能没有任何显着的影响。在标准优化函数和四个神经进化基准环境中的结果表明，实验运行时间减少了两个数量级的因素，将数天的计算变成了几分钟。更令人惊讶的是，我们观察到，减少了两个数量级的代数，从而具有显着更短的血统不影响QD算法的性能。这些结果表明，QD现在可以从硬件加速中受益，这对深度学习的蓬勃发展做出了重大贡献。
英文摘要：
Quality-Diversity (QD) optimization algorithms are a well-known approach to generate large collections of diverse and high-quality solutions. However, derived from evolutionary computation, QD algorithms are population-based methods which are known to be data-inefficient and requires large amounts of computational resources. This makes QD algorithms slow when used in applications where solution evaluations are computationally costly. A common approach to speed up QD algorithms is to evaluate solutions in parallel, for instance by using physical simulators in robotics. Yet, this approach is limited to several dozen of parallel evaluations as most physics simulators can only be parallelized more with a greater number of CPUs. With recent advances in simulators that run on accelerators, thousands of evaluations can now be performed in parallel on single GPU/TPU. In this paper, we present QDax, an accelerated implementation of MAP-Elites which leverages massive parallelism on accelerators to make QD algorithms more accessible. We show that QD algorithms are ideal candidates to take advantage of progress in hardware acceleration. We demonstrate that QD algorithms can scale with massive parallelism to be run at interactive timescales without any significant effect on the performance. Results across standard optimization functions and four neuroevolution benchmark environments shows that experiment runtimes are reduced by two factors of magnitudes, turning days of computation into minutes. More surprising, we observe that reducing the number of generations by two orders of magnitude, and thus having significantly shorter lineage does not impact the performance of QD algorithms. These results show that QD can now benefit from hardware acceleration, which contributed significantly to the bloom of deep learning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
