一阶方法中H-对偶的统一理论

中文摘要 AI 辅助

我们对光滑强凸优化与压缩不动点问题中的H-对偶性提供两种互补的解释。H-对偶性指的是，对于给定性能设置，许多固定步长一阶方法（FSFOMs）的最坏情况性能，恰好等于其“时间反转”对应方法及配对性能设置的最坏情况性能。该概念作为一种机制受到关注，可将具有最终次优性保证的算法自动转换为具有最终梯度范数保证的算法。自首次发现以来，虽已通过经验发现更多H-对偶性能设置配对，但这些现象的通用解释仍难以捉摸。我们对H-对偶性的第一种解释表明，在一类具有极值曲率特性的实例上，FSFOMs生成的最终迭代是初始迭代的标量倍，且该标量在FSFOM的时间反转下保持不变；结合许多FSFOMs的最坏情况为此类极值曲率实例的经验观察，这解释了H-对偶性为何常出现。第二种解释是互补的，利用FSFOMs收敛率的性能估计证书概念，我们展示了此类证书的显式变换，其保留了定义证书的诸多属性，且在许多数值与分析示例中，该变换可将算法收敛率的证书转换为其时间反转算法的证书。作为应用，我们证明了ITEM的H-对偶的收敛保证，与推测的最优梯度到梯度保证匹配。

英文摘要

We provide two complementary explanations of H-duality in smooth strongly convex optimization and contractive fixed-point problems. H-duality refers to the phenomenon where the worst-case performance of many fixed-step first-order methods (FSFOMs) for a given performance setup are exactly equal to the worst-case performance of their "time-reversed" counterparts and a paired performance setup. This concept gained interest as a mechanism for automatically converting an algorithm with guarantees on the final suboptimality to algorithms with guarantees on the final gradient norm. Since its initial discovery, additional H-dual performance setup pairings have been discovered empirically, yet a general explanation for these phenomena remained elusive. Our first explanation of H-duality shows that, on a class of instances with extremal curvature properties, FSFOMs produce a final iterate that is a scalar multiple of the initial iterate, and that this scalar is invariant under the time reversal of the underlying FSFOM. This, together with the empirical observation that many FSFOMs have such an instance with extremal curvature as their worst case, gives one explanation for why H-duality often occurs. The second explanation is complementary and uses the notion of a performance estimation certificate of the convergence rate of a FSFOM. We exhibit an explicit transformation of such certificates that preserves many of the properties that define such a certificate, and we show that in many numerical and analytical examples, this transformation in fact transforms certificates of the convergence rate of an algorithm into certificates for its time-reversal. As an application, we prove a convergence guarantee for the H-dual of ITEM matching the conjectured optimal gradient-to-gradient guarantee.

发表机构

• University of Waterloo（滑铁卢大学）
• Purdue University, Daniels School of Business（普渡大学丹尼尔斯商学院）

机构由 AI 辅助整理，请以论文原文为准。