在概率测度优化中的Nesterov加速

中文摘要 AI 辅助

在现代机器学习、科学计算和不确定性量化中，概率测度优化已成为日益重要的范式。受欧几里得空间中Nesterov加速梯度法启发，我们在概率测度空间\(\mathcal{P}_2\)上开发了重球法和Nesterov加速法，并建立了与欧几里得对应方法相匹配的非渐近收敛保证。特别地，我们推导了关于迭代次数和用于表示基础概率分布的粒子数的收敛率。将加速优化从欧几里得空间扩展到概率测度具有挑战性，因为自然的动量概念需要概率空间集的切丛等概念，且难以进行数值操作。为克服这些困难，我们引入了两种互补的提升过程。第一种通过哈密顿公式将概率测度提升到相空间，在动力学中引入动量变量。第二种将概率测度提升到一个公共希尔伯特空间，恢复收敛分析所需的线性结构，同时产生可执行的粒子动力学。这两种互补的提升过程共同为在概率测度空间上设计、分析和实现基于动量的加速优化方法提供了一种系统方法。

英文摘要

Optimization over probability measures has become an increasingly important paradigm in modern machine learning, scientific computing, and uncertainty quantification. Motivated by Nesterov's accelerated gradient method in Euclidean space, we develop Heavy-ball and Nesterov acceleration methods over the probability measure space $\mathcal{P}_2$ and establish non-asymptotic convergence guarantees that match their Euclidean counterparts. In particular, we derive convergence rates with respect to both the number of iterations and the number of particles used to represent the underlying probability distributions. Extending accelerated optimization from Euclidean space to probability measures is challenging. The natural notion of momentum requires concepts such as tangent bundles of the set of probability space and they are hard to operate numerically. To overcome these difficulties, we introduce two complementary lifting procedures. The first lifts probability measures to phase space through a Hamiltonian formulation, introducing momentum variables into the dynamics. The second lifts probability measures to a common Hilbert space, restoring the linear structure required for convergence analysis while simultaneously yielding executable particle dynamics. Together, these two complementary lifting procedures provide a systematic methodology for designing, analyzing, and implementing momentum-based accelerated optimization methods over probability measure spaces.

发表机构

• University of Wisconsin–Madison（威斯康星大学麦迪逊分校）
• University of California, Los Angeles（加利福尼亚大学洛杉矶分校）

机构由 AI 辅助整理，请以论文原文为准。