Bregman近端类型算法的伪平稳性和困难结果
Spurious Stationarity and Hardness Results for Bregman Proximal-Type Algorithms
AI总结：
本文揭示了Bregman近端类型算法（如镜像下降）在非欧几何下可能陷入伪平稳点，即使对于凸问题，若Bregman核的梯度非Lipschitz连续，停滞可无限持续，并指出该现象在非凸多面体约束问题中普遍存在，挑战了现有收敛性理论。
摘要：
Bregman近端类型算法（BPs），如镜像下降，已成为机器学习和数据科学中通过非欧几何利用问题结构的流行工具。在本文中，我们表明BPs可能被困在一类非平稳点附近，我们称之为\emph{伪平稳点}。如果Bregman核的梯度不是Lipschitz连续的，即使对于凸问题，这种停滞也可能持续任意有限次迭代。根本原因在于欧几里得几何和Bregman几何在下降行为上的根本对比：虽然欧几里得梯度下降确保在任何非平稳点附近充分下降，但BPs可能在伪平稳点附近表现出任意缓慢的下降。因此，常用的基于Bregman的平稳性度量，例如Bregman散度的相对变化，可能在伪平稳点附近消失。这可能误导性地表明收敛，即使迭代点仍远离任何真正的平稳点。我们的分析进一步揭示，伪平稳点并非病态，而是在具有多面体约束的广泛非凸问题类中普遍出现。综上所述，我们的发现揭示了基于Bregman的优化方法中的一个严重盲点，并呼吁新的理论工具和算法保障以确保可靠的收敛。
英文摘要：
Bregman proximal-type algorithms (BPs), such as mirror descent, have become popular tools in machine learning and data science for exploiting problem structures through non-Euclidean geometries. In this paper, we show that BPs can get trapped near a class of non-stationary points, which we term \emph{spurious stationary points}. Such stagnation can persist for any finite number of iterations if the gradient of the Bregman kernel is not Lipschitz continuous, even in convex problems. The root cause lies in a fundamental contrast in descent behavior between Euclidean and Bregman geometries: While Euclidean gradient descent ensures sufficient decrease near any non-stationary point, BPs may exhibit arbitrarily slow decrease around spurious stationary points. As a result, commonly used Bregman-based stationarity measure, such as relative change in terms of Bregman divergence, can vanish near spurious stationary points. This may misleadingly suggest convergence, even when the iterates remain far from any true stationary point. Our analysis further reveals that spurious stationary points are not pathological, but rather occur generically in a broad class of nonconvex problems with polyhedral constraints. Taken together, our findings reveal a serious blind spot in Bregman-based optimization methods and calls for new theoretical tools and algorithmic safeguards to ensure reliable convergence.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
