基于二阶流和变量分裂的若干加速且稳定的伪能量耗散拉格朗日乘子方法
Several Accelerated and Stable Pseudo-Energy-Dissipative Lagrange Multiplier Methods based on Second-order Flow and Variable Splitting
- Shandong University（山东大学）
- Eastern Institute of Technology（东方理工学院）
- The Chinese University of Hong Kong（香港中文大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
基于二阶流和变量分裂，提出两种加速且稳定的拉格朗日乘子优化方法，具有伪能量耗散和收敛保证，实验验证其能逃离局部极小并鲁棒于学习率选择。
AI中文摘要：
基于二阶惯性动力学，我们开发了两种加速的拉格朗日乘子（LM）优化方法：用于无约束非凸优化问题的线性LM二阶流方法和线性LM变量与算子分裂方法。提出了两种方法的松弛和自适应变体，以防止拉格朗日乘子退化并实现时间步长的自动调整。在新的二阶LM优化框架内，证明了相关的伪能量泛函沿迭代轨迹单调耗散，并严格建立了生成的迭代序列收敛到稳定点。在函数优化和偏微分方程基准问题上的大量数值实验，包括作为物理信息神经网络（PINNs）和深度算子网络（DeepONets）优化器的应用，表明所提出的方法能有效逃离局部最小值，实现高精度，并在神经网络训练中对初始学习率的选择表现出强鲁棒性。
英文摘要：
Based on second-order inertial dynamics, we develop two accelerated Lagrange multiplier (LM)-based optimization methods: Linear LM-based second-order flow method and linear LM-based variable and operator splitting method for unconstrained non-convex optimization problems. Relaxed and adaptive variants of both methods are proposed to prevent degeneration of the Lagrange multiplier and to enable automatic adjustment of the time step size. Within the novel second-order LM-based optimization framework, the associated pseudo-energy functional is shown to dissipate monotonically along the iterative trajectory, and convergence of the generated iterates to stationary points is rigorously established. Extensive numerical experiments on function optimization and partial differential equation benchmark problems, including applications as optimizers for physics-informed neural networks (PINNs) and deep operator networks (DeepONets), demonstrate that the proposed methods effectively escape local minima, achieve high accuracy, and exhibit strong robustness with respect to the choice of the initial learning rate during neural network training.
