通过 Bregman 近端梯度算法最小化正则化 Rényi 散度
Regularized Rényi divergence minimization through Bregman proximal gradient algorithms
AI总结：
本文研究了指数族上正则化 Rényi 散度最小化的变分推断问题，提出确定性与基于采样的随机 Bregman 近端梯度算法，证明其具有强收敛保证，并在数值实验中表现出通用、鲁棒和竞争力。
摘要：
我们研究了在指数族上最小化正则化 Rényi 散度的变分推断问题。我们提出用 Bregman 近端梯度算法来解决该问题。我们提出了一种基于采样的算法以覆盖黑盒设置，这对应于带有偏梯度估计器的随机 Bregman 近端梯度算法。我们证明了所得算法可被视为带有额外近端步骤的松弛矩匹配算法。使用 Bregman 更新代替欧几里得更新使我们能够利用近似模型的几何结构。利用这一视角，我们为确定性和随机算法均证明了强收敛保证，包括目标函数的单调递减、收敛到驻点或最小化器，以及几何收敛速率。正如数值实验所证明的那样，这些新理论见解带来了一种通用、鲁棒且具竞争力的方法。
英文摘要：
We study the variational inference problem of minimizing a regularized Rényi divergence over an exponential family. We propose to solve this problem with a Bregman proximal gradient algorithm. We propose a sampling-based algorithm to cover the black-box setting, corresponding to a stochastic Bregman proximal gradient algorithm with biased gradient estimator. We show that the resulting algorithms can be seen as relaxed moment-matching algorithms with an additional proximal step. Using Bregman updates instead of Euclidean ones allows us to exploit the geometry of our approximate model. We prove strong convergence guarantees for both our deterministic and stochastic algorithms using this viewpoint, including monotonic decrease of the objective, convergence to a stationary point or to the minimizer, and geometric convergence rates. These new theoretical insights lead to a versatile, robust, and competitive method, as illustrated by numerical experiments.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
