arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2210.12573cs.LGmath.OC
收藏

一种利用Hessian对称性的高效非线性加速方法

An Efficient Nonlinear Acceleration method that Exploits Symmetry of the Hessian

Harvard University(哈佛大学);Emory University(埃默里大学);University of Minnesota(明尼苏达大学)

Huan He, Shifan Zhao, Ziyuan Tang, Joyce C Ho, Yousef Saad, Yuanzhe Xi

摘要:

非线性加速方法是加速定点迭代的强大技术。然而,许多加速方法需要存储大量先前的迭代,如果计算资源有限,这可能变得不切实际。在本文中,我们提出了一个非线性截断广义共轭残差方法(nlTGCR),其目标是利用对称的海森,以减少内存的使用。所提出的方法可以解释为不精确牛顿或拟牛顿方法。我们表明,在全局策略的帮助下,如残差检查技术,nlTGCR可以全局收敛的一般非线性问题,并在温和的条件下,nlTGCR能够实现超线性收敛。我们进一步分析了nlTGCR在随机环境下的收敛性。数值结果表明,nlTGCR的优越性相比,与其他几个有竞争力的基线方法对一些问题。我们的代码将在未来可用。

英文摘要:

Nonlinear acceleration methods are powerful techniques to speed up fixed-point iterations. However, many acceleration methods require storing a large number of previous iterates and this can become impractical if computational resources are limited. In this paper, we propose a nonlinear Truncated Generalized Conjugate Residual method (nlTGCR) whose goal is to exploit the symmetry of the Hessian to reduce memory usage. The proposed method can be interpreted as either an inexact Newton or a quasi-Newton method. We show that, with the help of global strategies like residual check techniques, nlTGCR can converge globally for general nonlinear problems and that under mild conditions, nlTGCR is able to achieve superlinear convergence. We further analyze the convergence of nlTGCR in a stochastic setting. Numerical results demonstrate the superiority of nlTGCR when compared with several other competitive baseline approaches on a few problems. Our code will be available in the future.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑