基于新颖性搜索和发射器的稀疏奖励探索
Sparse Reward Exploration via Novelty Search and Emitters
摘要：
基于奖励的优化算法既需要探索，以找到奖励，又需要利用，以最大限度地提高性能。在稀疏奖励设置中，对有效探索的需求甚至更加重要，在稀疏奖励设置中，性能反馈很少给出，因此使其不适合用于指导搜索过程。在这项工作中，我们介绍了SparsE奖励探索通过新奇和发射器（SERENE）算法，能够有效地探索一个搜索空间，以及优化奖励发现在潜在的不同领域。与现有的基于发射器的方法相反，SERENE将搜索空间探索和奖励利用分成两个交替的过程。第一个过程通过新颖性搜索（一种发散搜索算法）执行探索。第二种是通过发射器利用发现的奖励区域，即基于群体的优化算法的本地实例。元调度器通过在两个进程之间交替来分配全局计算预算，确保发现和有效利用不相交的奖励区域。SERENE返回覆盖搜索空间的不同解决方案的集合，以及每个不同奖励区域的高性能解决方案的集合。我们在各种稀疏奖励环境中评估SERENE，并显示它与现有基线相比是有利的。
英文摘要：
Reward-based optimization algorithms require both exploration, to find rewards, and exploitation, to maximize performance. The need for efficient exploration is even more significant in sparse reward settings, in which performance feedback is given sparingly, thus rendering it unsuitable for guiding the search process. In this work, we introduce the SparsE Reward Exploration via Novelty and Emitters (SERENE) algorithm, capable of efficiently exploring a search space, as well as optimizing rewards found in potentially disparate areas. Contrary to existing emitters-based approaches, SERENE separates the search space exploration and reward exploitation into two alternating processes. The first process performs exploration through Novelty Search, a divergent search algorithm. The second one exploits discovered reward areas through emitters, i.e. local instances of population-based optimization algorithms. A meta-scheduler allocates a global computational budget by alternating between the two processes, ensuring the discovery and efficient exploitation of disjoint reward areas. SERENE returns both a collection of diverse solutions covering the search space and a collection of high-performing solutions for each distinct reward area. We evaluate SERENE on various sparse reward environments and show it compares favorably to existing baselines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
