在基于比较的步长自适应随机搜索中正齐函数的线性收敛性：具有广义五分之一成功规则的(1+1)ES
Linear Convergence on Positively Homogeneous Functions of a Comparison Based Step-Size Adaptive Randomized Search: the (1+1) ES with Generalized One-fifth Success Rule
AI总结：
本文研究了基于比较的步长自适应随机搜索在正齐函数上的全局线性收敛性，证明了(1+1)ES在广义五分之一成功规则下的线性收敛性，适用于非准凸和非连续函数。
摘要：
在无约束数值优化的背景下，本文研究了一种简单的概率无导数优化算法（DFO）的全局线性收敛性。该算法从标准多元正态分布缩放后的步长中采样候选解，并以当前解为中心。如果该解的目标函数值优于当前解，则被接受。算法的关键在于通过维持一定成功概率来调整步长。该算法是Rechenberg提出的著名的(1+1)进化策略（ES）在五分之一成功规则下的推广，也被称为复合随机搜索或步长自适应随机搜索。除了无导数外，该算法还是无函数值的：它仅通过比较来利用目标函数。它属于基于比较的步长自适应随机搜索（CB-SARS）类。对于收敛分析，我们遵循一篇同行论文中开发的方法，用于研究CB-SARS的线性收敛性：通过利用算法的不变性性质，将研究全局线性收敛性在缩放不变函数上的问题转化为研究其下归一化的马尔可夫链（MC）的稳定性。因此，我们通过研究归一化MC的稳定性（不可约性、可 recurrence、正性、几何遍历性）来证明全局线性收敛性。更具体地说，我们证明从任何初始解和任何步长开始，以概率一和期望发生线性收敛。我们的证明适用于单峰函数，即严格递增函数与正齐函数（度数α）的复合（假设也是连续可微的）。该函数类包括范数函数的复合，也包括非准凸函数。由于通过严格递增函数的复合，它包括非连续函数。我们发现全局线性收敛的充分条件是在线性函数上增加步长，这通常是标准参数选择下满足的条件。尽管早在40多年前被引入，我们在此提供了(1+1)-ES在广义五分之一成功规则下的全球线性收敛性证明，以及在包含非准凸和非连续函数的此类函数类上的CB-SARS线性收敛性证明。我们的证明也适用于之前已证明某些CB-SARS线性收敛性的函数，即凸二次函数（包括著名的球函数）。
英文摘要：
In the context of unconstraint numerical optimization, this paper investigates the global linear convergence of a simple probabilistic derivative-free optimization algorithm (DFO). The algorithm samples a candidate solution from a standard multivariate normal distribution scaled by a step-size and centered in the current solution. This solution is accepted if it has a better objective function value than the current one. Crucial to the algorithm is the adaptation of the step-size that is done in order to maintain a certain probability of success. The algorithm, already proposed in the 60's, is a generalization of the well-known Rechenberg's $(1+1)$ Evolution Strategy (ES) with one-fifth success rule which was also proposed by Devroye under the name compound random search or by Schumer and Steiglitz under the name step-size adaptive random search. In addition to be derivative-free, the algorithm is function-value-free: it exploits the objective function only through comparisons. It belongs to the class of comparison-based step-size adaptive randomized search (CB-SARS). For the convergence analysis, we follow the methodology developed in a companion paper for investigating linear convergence of CB-SARS: by exploiting invariance properties of the algorithm, we turn the study of global linear convergence on scaling-invariant functions into the study of the stability of an underlying normalized Markov chain (MC). We hence prove global linear convergence by studying the stability (irreducibility, recurrence, positivity, geometric ergodicity) of the normalized MC associated to the $(1+1)$-ES. More precisely, we prove that starting from any initial solution and any step-size, linear convergence with probability one and in expectation occurs. Our proof holds on unimodal functions that are the composite of strictly increasing functions by positively homogeneous functions with degree $α$ (assumed also to be continuously differentiable). This function class includes composite of norm functions but also non-quasi convex functions. Because of the composition by a strictly increasing function, it includes non continuous functions. We find that a sufficient condition for global linear convergence is the step-size increase on linear functions, a condition typically satisfied for standard parameter choices. While introduced more than 40 years ago, we provide here the first proof of global linear convergence for the $(1+1)$-ES with generalized one-fifth success rule and the first proof of linear convergence for a CB-SARS on such a class of functions that includes non-quasi convex and non-continuous functions. Our proof also holds on functions where linear convergence of some CB-SARS was previously proven, namely convex-quadratic functions (including the well-know sphere function).
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
