广义Polyak-Chaojasiewicz函数的块梯度法的整体收敛
Global Convergence of Arbitrary-Block Gradient Methods for Generalized Polyak-Łojasiewicz Functions
摘要：
在本文中，我们介绍了两个新的推广理论的梯度下降型方法在最近的设置。首先，我们引入比例函数，我们进一步使用它来分析所有已知的（和许多新的）块坐标下降法在一个单一的框架下的块选择规则。该框架包括具有均匀、非均匀甚至自适应采样策略的随机方法，以及具有批量、贪婪或循环选择规则的确定性方法。其次，强凸优化理论最近被推广到一类满足所谓的Polyak条件的非凸函数。为了反映这一推广的弱凸的情况下，我们引入弱Polyak-Chaojasiewicz条件，使用它我们给整体收敛保证一类非凸函数以前没有考虑在理论上。此外，我们建立（必然有点弱）收敛保证更大的一类非凸函数满足一定的光滑性假设。通过结合上述两个概括，我们恢复了最先进的收敛保证的一个大类以前已知的方法和设置作为我们的一般框架的特殊情况。此外，我们的框架允许衍生新的保证，许多新的组合的方法和设置，以及一大类新的非凸目标。我们的方法的灵活性为未来的研究提供了很大的潜力，作为一个新的块选择程序将有一个收敛的保证，我们的框架中考虑的所有目标，而根据我们的方法分析的一个新的目标将有一个完整的舰队块选择规则与收敛保证随时可用。
英文摘要：
In this paper we introduce two novel generalizations of the theory for gradient descent type methods in the proximal setting. First, we introduce the proportion function, which we further use to analyze all known (and many new) block-selection rules for block coordinate descent methods under a single framework. This framework includes randomized methods with uniform, non-uniform or even adaptive sampling strategies, as well as deterministic methods with batch, greedy or cyclic selection rules. Second, the theory of strongly-convex optimization was recently generalized to a specific class of non-convex functions satisfying the so-called Polyak-Łojasiewicz condition. To mirror this generalization in the weakly convex case, we introduce the Weak Polyak-Łojasiewicz condition, using which we give global convergence guarantees for a class of non-convex functions previously not considered in theory. Additionally, we establish (necessarily somewhat weaker) convergence guarantees for an even larger class of non-convex functions satisfying a certain smoothness assumption only. By combining the two abovementioned generalizations we recover the state-of-the-art convergence guarantees for a large class of previously known methods and setups as special cases of our general framework. Moreover, our frameworks allows for the derivation of new guarantees for many new combinations of methods and setups, as well as a large class of novel non-convex objectives. The flexibility of our approach offers a lot of potential for future research, as a new block selection procedure will have a convergence guarantee for all objectives considered in our framework, while a new objective analyzed under our approach will have a whole fleet of block selection rules with convergence guarantees readily available.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
