发表机构
Weierstrass Institute for Applied Analysis and Stochastics; Physikalisch-Technische Bundesanstalt (PTB); Technical University of Berlin（魏尔斯特拉斯应用分析与随机研究所; 德国联邦物理技术研究院; 柏林工业大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
研究单边赫尔德正则性下连续可微非凸目标的自适应梯度下降，提出基于正单边赫尔德曲率估计的自适应标量步长法，结合充分下降保障，在两个全批量基准测试中表现良好，证明单边赫尔德曲率是有效自适应步长信号。
AI 中文摘要
我们研究了在单边赫尔德正则性下，针对连续可微、可能非凸目标函数的自适应梯度下降。与控制全梯度变化的经典赫尔德或利普希茨梯度假设不同，我们的条件仅限制下降不等式中出现的方向项。当大梯度变化与更新方向正交或有利于更新方向时，这可以允许采用不太保守的步长。我们提出了一种基于正单边赫尔德曲率估计的自适应标量步长方法，并结合了一个简单的充分下降保障。对于包含已接受更新段的凸区域上的非凸目标函数，我们证明了一个明确的最佳迭代平稳性界，其速率由赫尔德指数确定。与预定的递减步长方案不同，该方法适应局部下降几何结构。我们在两个旨在区分方向曲率和全梯度变化的全批量基准上评估了该方法。在一个二元分类问题上，该方法在比较的标量梯度方法中实现了最低的最终交叉熵、目标值和梯度范数，以及最大的分类间隔。在一个非凸赫尔德回归问题上，它获得了最低的最终目标差距和梯度范数。这些结果表明，当全梯度变化因不阻碍下降的方向而膨胀时，单边赫尔德曲率是一个有效的自适应步长信号。
英文摘要
We study adaptive gradient descent for continuously differentiable, possibly nonconvex objectives under one-sided Hölder regularity. Unlike classical Hölder- or Lipschitz-gradient assumptions, which control the full gradient variation, our condition bounds only the directional term appearing in the descent inequality. This can allow less conservative step sizes when large gradient changes are orthogonal to, or favorable along, the update direction. We propose an adaptive scalar-step method based on an estimate of positive one-sided Hölder curvature, combined with a simple sufficient-decrease safeguard. For nonconvex objectives on a convex region containing the accepted update segments, we prove an explicit best-iterate stationarity bound with a rate determined by the Hölder exponent. Unlike predetermined diminishing step-size schemes, the method adapts to the local descent geometry. We evaluate the approach on two full-batch benchmarks designed to separate directional curvature from full gradient variation. On a binary classification problem, the method achieves the lowest final cross-entropy, objective value, and gradient norm, together with the largest classification margin among the compared scalar gradient methods. On a nonconvex Hölder regression problem, it attains the lowest final objective gap and gradient norm. These results indicate that one-sided Hölder curvature is an effective adaptive step-size signal when full-gradient variation is inflated by directions that do not hinder descent.
