变分相干函数的无参数随机最优化
Parameter-free Stochastic Optimization of Variationally Coherent Functions
摘要：
我们设计和分析了一个算法的一个大类函数的一阶随机最优化的$\mathbb{R}^d$。特别地，我们考虑的\emph{variationally coherent}功能，可以是凸的或非凸的。我们的算法在变分相干函数上的迭代几乎必然收敛到全局极小点$\boldsymbol{x}^*$。此外，在$T$迭代之后，具有相同超参数的非常相同的算法保证凸函数上的期望次优性间隙对于任何$ε>0$都由$\widetilde{O}(\|\boldsymbol{x}^* - \boldsymbol{x}_0\| T^{-1/2+ε})$限定。它是第一个同时实现这两个属性的算法。此外，凸函数的速率基本上与无参数算法的性能相匹配。我们的算法是Follow The Regularized Leader算法的一个实例，并增加了使用\emph{rescaled gradients}和时变线性正则化器的扭曲。
英文摘要：
We design and analyze an algorithm for first-order stochastic optimization of a large class of functions on $\mathbb{R}^d$. In particular, we consider the \emph{variationally coherent} functions which can be convex or non-convex. The iterates of our algorithm on variationally coherent functions converge almost surely to the global minimizer $\boldsymbol{x}^*$. Additionally, the very same algorithm with the same hyperparameters, after $T$ iterations guarantees on convex functions that the expected suboptimality gap is bounded by $\widetilde{O}(\|\boldsymbol{x}^* - \boldsymbol{x}_0\| T^{-1/2+ε})$ for any $ε>0$. It is the first algorithm to achieve both these properties at the same time. Also, the rate for convex functions essentially matches the performance of parameter-free algorithms. Our algorithm is an instance of the Follow The Regularized Leader algorithm with the added twist of using \emph{rescaled gradients} and time-varying linearithmic regularizers.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
