arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2205.07544math.OC
收藏

梯度中带加性噪声的非凸问题的梯度法的停止规则

Stopping Rules for Gradient Methods for Non-Convex Problems with Additive Noise in Gradient

Boris T. Polyak, Ilia A. Kuruzov, Fedor S. Stonyakin

摘要:

我们研究的梯度法的假设下,一个可加的不精确梯度,一般来说,非凸问题。目标函数的非凸性以及在迭代时使用不精确的指定梯度会导致各种问题。例如,梯度法的轨迹可以离起始点足够远。另一方面,在存在噪声的情况下对梯度法的轨迹的无界去除可以导致从期望的精确解去除该方法的轨迹。在梯度不精确的假设和梯度占优的条件下,得到了研究梯度法轨迹行为的结果。这是众所周知的,这样的条件是有效的许多重要的非凸问题。此外,它导致良好的复杂性保证的梯度法。提出了一种梯度法的提前终止准则。首先,它保证在函数方面实现方法的出口点的可接受的质量。其次,停止规则确保该点与所选初始位置的距离相当适中。除了具有恒定步长的梯度法之外,还详细研究了具有自适应步长的其变体,这使得在梯度的Lipschitz常数未知的情况下应用所开发的技术成为可能。一些计算实验已经进行了研究梯度法,证明了建议的停止规则的有效性。

英文摘要:

We study the gradient method under the assumption that an additively inexact gradient is available for, generally speaking, non-convex problems. The non-convexity of the objective function, as well as the use of an inexactness specified gradient at iterations, can lead to various problems. For example, the trajectory of the gradient method may be far enough away from the starting point. On the other hand, the unbounded removal of the trajectory of the gradient method in the presence of noise can lead to the removal of the trajectory of the method from the desired exact solution. The results of investigating the behavior of the trajectory of the gradient method are obtained under the assumption of the inexactness of the gradient and the condition of gradient dominance. It is well known that such a condition is valid for many important non-convex problems. Moreover, it leads to good complexity guarantees for the gradient method. A rule of early stopping of the gradient method is proposed. Firstly, it guarantees achieving an acceptable quality of the exit point of the method in terms of the function. Secondly, the stopping rule ensures a fairly moderate distance of this point from the chosen initial position. In addition to the gradient method with a constant step, its variant with adaptive step size is also investigated in detail, which makes it possible to apply the developed technique in the case of an unknown Lipschitz constant for the gradient. Some computational experiments have been carried out which demonstrate effectiveness of the proposed stopping rule for the investigated gradient methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑