Optimization with Momentum: Dynamical, Control-Theoretic, and Symplectic Perspectives

关于辛加速优化的实用视角

Practical Perspectives on Symplectic Accelerated Optimization

本文探讨了辛加速优化算法的实用改进方法，通过动量重启方案减少振荡、时间循环避免数值精度不稳定，并比较几何积分技术以简化调参，显著提升了算法的计算效率、稳定性和鲁棒性。

深度学习中的辛方法

Symplectic Methods in Deep Learning

本文研究深度学习中的辛网络架构，基于高阶显式方法构造具有非消失梯度性质的辛网络，以保证数值稳定性，并在多个示例上验证其效率。

The long time behavior and the rate of convergence of symplectic convex algorithms obtained via splitting discretizations of inertial damping systems

In this paper we propose new numerical algorithms in the setting of unconstrained optimization problems and we study the rate of convergence in the iterates of the objective function. Furthermore, our algorithms are based upon splitting and symplectic methods and they preserve the energy properties of the inherent continuous dynamical system that contains a Hessian perturbation. At the same time, we show that Nesterov gradient method is equivalent to a Lie-Trotter splitting applied to a Hessian driven damping system. Finally, some numerical experiments are presented in order to validate the th…

Symplectic eigenvalue problem via trace minimization and Riemannian optimization

We address the problem of computing the smallest symplectic eigenvalues and the corresponding eigenvectors of symmetric positive-definite matrices in the sense of Williamson's theorem. It is formulated as minimizing a trace cost function over the symplectic Stiefel manifold. We first investigate various theoretical aspects of this optimization problem such as characterizing the sets of critical points, saddle points, and global minimizers as well as proving that non-global local minimizers do not exist. Based on our recent results on constructing Riemannian structures on the symplectic Stiefel…

求解一般非单调包含问题的辛外梯度型方法

Symplectic Extra-gradient Type Method for Solving General Non-monotone Inclusion Problem

针对现有加速外梯度方法直接用初始点易降低收敛速度的局限，提出基于辛加速技术的新型加速外梯度方法，建立收敛性理论并引入线搜索提升效率，数值实验验证其收敛速度优于现有同类方法。

Symplectic discrete-time energy-based control for nonlinear mechanical systems

In this article we present a novel discrete-time design approach which reduces the deteriorating effects of sampling on stability and performance in digitally controlled nonlinear mechanical systems. The method is motivated by recent results for linear systems, where feedback imposes closed-loop behavior that exactly represents the symplectic discretization of a desired target system. In the nonlinear case, both the second order accurate representation of the sampling process and the definition of the target dynamics stem from the application of the implicit midpoint rule. The implicit nature …