arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2103.01578cs.NE
收藏

凸二次函数上步长自适应的(1+1)-演化策略的收敛速度

Convergence Rate of the (1+1)-Evolution Strategy with Success-Based Step-Size Adaptation on Convex Quadratic Functions

Daiki Morinaga, Kazuto Fukuchi, Jun Sakuma, Youhei Akimoto

摘要:

基于一般凸二次函数及其单调变换,即$f(x) = g((x - x^*)^\mathrm{T} H (x - x^*))$,分析了基于成功的步长自适应(1+1)-演化策略,其中$g:\mathbb{R}\to\mathbb{R}$是严格增函数,$H$是正定对称矩阵,$x^* \in \mathbb{R}^d$是$f$的最优解.收敛速度,即从搜索点$m_t$到最优解$x^*$的距离的减小速度,被证明是在$O(\exp( - L / \mathrm{Tr}(H) ))$,其中$L$是$H$的最小特征值,$\mathrm{Tr}(H)$是$H$的迹。这个结果推广了已知的$O(\exp(- 1/d ))$对于$H = I_{d}$($I_d$是维度$d$的单位矩阵)的情况的速率和$O(\exp(- 1/ (d\cdotξ) ))$对于$H = \mathrm{diag}(ξ\cdot I_{d/2}, I_{d/2})$的情况的速率。据我们所知,这是第一个研究中,(1+1)-ES的收敛速度推导明确和严格的一般凸二次函数,它描绘了在Hessian $H$的特征值分布的影响,而不仅仅是$H$的条件数的影响。

英文摘要:

The (1+1)-evolution strategy (ES) with success-based step-size adaptation is analyzed on a general convex quadratic function and its monotone transformation, that is, $f(x) = g((x - x^*)^\mathrm{T} H (x - x^*))$, where $g:\mathbb{R}\to\mathbb{R}$ is a strictly increasing function, $H$ is a positive-definite symmetric matrix, and $x^* \in \mathbb{R}^d$ is the optimal solution of $f$. The convergence rate, that is, the decrease rate of the distance from a search point $m_t$ to the optimal solution $x^*$, is proven to be in $O(\exp( - L / \mathrm{Tr}(H) ))$, where $L$ is the smallest eigenvalue of $H$ and $\mathrm{Tr}(H)$ is the trace of $H$. This result generalizes the known rate of $O(\exp(- 1/d ))$ for the case of $H = I_{d}$ ($I_d$ is the identity matrix of dimension $d$) and $O(\exp(- 1/ (d\cdotξ) ))$ for the case of $H = \mathrm{diag}(ξ\cdot I_{d/2}, I_{d/2})$. To the best of our knowledge, this is the first study in which the convergence rate of the (1+1)-ES is derived explicitly and rigorously on a general convex quadratic function, which depicts the impact of the distribution of the eigenvalues in the Hessian $H$ on the optimization and not only the impact of the condition number of $H$.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑