软拟牛顿法：通过松弛割线约束保证正定性
Soft quasi-Newton: Guaranteed positive definiteness by relaxing the secant constraint
AI总结：
针对有界噪声下的优化问题，提出 soft QN，通过惩罚项松弛割线条件并保证 Hessian 逆近似正定，证明其线性收敛及鲁棒优越性能。
摘要：
我们提出了一种称为软拟牛顿法（soft QN）的新颖算法，用于存在有界噪声时的优化。传统拟牛顿算法容易受到此类扰动的影响。为了开发一种更鲁棒的拟牛顿方法，我们将 Hessian 更新的矩阵优化问题中的割线条件替换为其目标函数中的一个惩罚项，并推导出一个闭式更新公式。我们方法的一个关键特征是其能够保持 Hessian 逆近似的正定性。此外，我们建立了 soft QN 的以下性质：它在特定极限下恢复为 BFGS 方法，它对正曲率和负曲率一视同仁，并且它具有尺度不变性。这些特征共同增强了 soft QN 在噪声环境中的有效性。对于强凸目标函数和使用 soft QN 获得的 Hessian 近似，我们开发了一种算法，即使梯度和函数评估受到有界扰动，该算法也能向最优解的某个邻域呈现线性收敛。通过数值实验，我们证明了在各种场景下 soft QN 相比最先进方法具有优越性能。
英文摘要：
We propose a novel algorithm, termed soft quasi-Newton (soft QN), for optimization in the presence of bounded noise. Traditional quasi-Newton algorithms are vulnerable to such perturbations. To develop a more robust quasi-Newton method, we replace the secant condition in the matrix optimization problem for the Hessian update with a penalty term in its objective and derive a closed-form update formula. A key feature of our approach is its ability to maintain positive definiteness of the Hessian inverse approximation. Furthermore, we establish the following properties of soft QN: it recovers the BFGS method under specific limits, it treats positive and negative curvature equally, and it is scale invariant. Collectively, these features enhance the efficacy of soft QN in noisy environments. For strongly convex objective functions and Hessian approximations obtained using soft QN, we develop an algorithm that exhibits linear convergence toward a neighborhood of the optimal solution, even if gradient and function evaluations are subject to bounded perturbations. Through numerical experiments, we demonstrate superior performance of soft QN compared to state-of-the-art methods in various scenarios.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
