关于在优化中应用计算除法差分的一些注记
Some notes on applying computational divided differencing in optimization
AI总结:
本文探讨了在小扰动下准确计算有限差分的方法,提出计算除法差分技术以提高精度,并指出其在优化中的应用及局限性。
摘要:
我们考虑了当‖s‖非常小时准确计算有限差分f(x+s)-f(x)的问题。直接计算此差值在浮点算术中会因消去误差而失效,当s足够小时会得到0。然而,许多优化算法需要准确计算此有限差分以进行'充分减少'测试。Reps和Rall提出了一种称为'计算除法差分'的程序变换,类似于自动微分,用于高精度计算这些差分。计算差分的时间是计算f(x)时间的常数倍。然而,与自动微分不同,该技术由于分支代码(即'if'语句)的困难而不具备完全通用性。我们对计算除法差分在优化中的应用进行了若干评论。一点是该技术可以有效地用作停滞测试。
英文摘要:
We consider the problem of accurate computation of the finite difference $f(\x+\s)-f(\x)$ when $\Vert\s\Vert$ is very small. Direct evaluation of this difference in floating point arithmetic succumbs to cancellation error and yields 0 when $\s$ is sufficiently small. Nonetheless, accurate computation of this finite difference is required by many optimization algorithms for a "sufficient decrease" test. Reps and Rall proposed a programmatic transformation called "computational divided differencing" reminiscent of automatic differentiation to compute these differences with high accuracy. The running time to compute the difference is a small constant multiple of the running time to compute $f$. Unlike automatic differentiation, however, the technique is not fully general because of a difficulty with branching code (i.e., `if' statements). We make several remarks about the application of computational divided differencing to optimization. One point is that the technique can be used effectively as a stagnation test.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。