非线性共轭梯度法中独立性损失的检测与校正
Detecting and correcting the loss of independence in nonlinear conjugate gradient
摘要：
众所周知，非线性共轭梯度（CG）中的搜索方向有时可能变得几乎相关，从而导致收敛速度急剧减慢。我们对这种独立性的丧失进行了理论分析。该分析适用于一个严格凸的目标函数的情况下，是出于旧的工作Nemirovsky和Yudin。失去独立性会影响非线性CG的几个著名变体，包括Fletcher-Reeves，Polak-Ribière（非负变体）和Hager-Zhang。基于我们的分析，我们提出了一个相对便宜的计算测试检测独立性的损失。我们还提出了一种方法来纠正它，当它被检测到，我们称之为“子空间优化。“虽然校正方法有点昂贵，但我们的实验表明，在某些情况下，通常是最病态的情况下，它产生的方法比这三种变体中的任何一种都要快得多。即使我们的理论只涵盖强凸目标函数，我们提供的计算结果表明，检测和校正机制也可能持有承诺的非凸优化。
英文摘要：
It is well known that search directions in nonlinear conjugate gradient (CG) can sometimes become nearly dependent, causing a dramatic slow-down in the convergence rate. We provide a theoretical analysis of this loss of independence. The analysis applies to the case of a strictly convex objective function and is motivated by older work of Nemirovsky and Yudin. Loss of independence can affect several of the well-known variants of nonlinear CG including Fletcher-Reeves, Polak-Ribière (nonnegative variant), and Hager-Zhang. Based on our analysis, we propose a relatively inexpensive computational test for detecting loss of independence. We also propose a method for correcting it when it is detected, which we call "subspace optimization." Although the correction method is somewhat expensive, our experiments show that in some cases, usually the most ill-conditioned ones, it yields a method much faster than any of these three variants. Even though our theory covers only strongly convex objective functions, we provide computational results to indicate that the detection and correction mechanisms may also hold promise for nonconvex optimization.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
