DALex:通过多样聚合实现类词典式选择
DALex: Lexicase-like Selection via Diverse Aggregation
AI总结:
DALex通过随机加权聚合训练误差,将词典式选择转化为矩阵乘法,实现显著加速,同时保持几乎相同的求解性能,并可调节超参数在严格与松弛变体间插值。
摘要:
词典式选择已被证明在进化计算和机器学习的多个领域中比其他选择算法具有优势。在其标准形式中,词典式选择基于随机排序的训练案例(一次考虑一个)来过滤种群或其他集合。这种迭代过滤过程可能非常耗时,尤其是在训练案例数量庞大的场景中。在本文中,我们提出了一种新方法,该方法在选择个体方面几乎等同于词典式选择,但执行速度显著更快。这种新方法称为DALex(多样聚合词典式选择),它根据训练案例误差的加权和来选择最佳个体,其中权重是随机采样的。这使得我们可以将选择所需的核心计算表述为矩阵乘法,而不是递归的比较循环,从而可以利用为矩阵乘法设计的优化和并行算法来实现加速。此外,我们表明,通过调整一个名为“特殊性压力”的超参数(该参数表示每个单独训练案例所赋予的重要性),我们可以在词典式选择的行为与其“松弛”变体(如epsilon或批量词典式选择)之间进行插值。在程序合成、深度学习、符号回归和学习分类器系统上的结果表明,DALex在保持几乎相同的问题解决性能的同时,实现了相对于词典式选择及其松弛变体的显著加速。在固定的计算预算下,这些节省释放了资源,可用于增加种群规模或代数,从而有可能解决更困难的问题。
英文摘要:
Lexicase selection has been shown to provide advantages over other selection algorithms in several areas of evolutionary computation and machine learning. In its standard form, lexicase selection filters a population or other collection based on randomly ordered training cases that are considered one at a time. This iterated filtering process can be time-consuming, particularly in settings with large numbers of training cases. In this paper, we propose a new method that is nearly equivalent to lexicase selection in terms of the individuals that it selects, but which does so significantly more quickly. The new method, called DALex (for Diversely Aggregated Lexicase), selects the best individual with respect to a weighted sum of training case errors, where the weights are randomly sampled. This allows us to formulate the core computation required for selection as matrix multiplication instead of recursive loops of comparisons, which in turn allows us to take advantage of optimized and parallel algorithms designed for matrix multiplication for speedup. Furthermore, we show that we can interpolate between the behavior of lexicase selection and its "relaxed" variants, such as epsilon or batch lexicase selection, by adjusting a single hyperparameter, named "particularity pressure," which represents the importance granted to each individual training case. Results on program synthesis, deep learning, symbolic regression, and learning classifier systems demonstrate that DALex achieves significant speedups over lexicase selection and its relaxed variants while maintaining almost identical problem-solving performance. Under a fixed computational budget, these savings free up resources that can be directed towards increasing population size or the number of generations, enabling the potential for solving more difficult problems.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。