全局收敛的变分推断
Globally Convergent Variational Inference
AI总结：
该研究针对变分推断中ELBO仅能保证局部收敛的问题，证明了基于前向KL散度的神经后验估计方法的全局收敛性，且实验表明其效果优于基于ELBO的优化。
摘要：
在变分推断（VI）中，后验分布的近似值通过数值优化从一族分布中选取。使用最常见的变分目标函数——即证据下界（ELBO）时，仅能保证收敛到局部最优解。在本研究中，我们转而证明了一种特定VI方法的全局收敛性。该VI方法可视为神经后验估计（NPE）的一个实例，它通过最小化包含（前向）KL散度的期望，来拟合由神经网络参数化的变分分布。我们的收敛结果依赖神经切线核（NTK）来刻画在函数空间中考虑变分目标时产生的梯度动力学。在固定正定神经切线核的渐近情形下，我们推导了使变分目标在再生核希尔伯特空间（RKHS）中存在唯一解的条件，随后证明函数空间中的梯度下降动力学会收敛到该唯一函数。在消融研究和实际问题中，我们证明了我们的结果能够解释NPE在非渐近有限神经元设置下的行为，并且NPE的表现优于常收敛到浅层局部最优的基于ELBO的优化方法。
英文摘要：
In variational inference (VI), an approximation of the posterior distribution is selected from a family of distributions through numerical optimization. With the most common variational objective function, known as the evidence lower bound (ELBO), only convergence to a local optimum can be guaranteed. In this work, we instead establish the global convergence of a particular VI method. This VI method, which may be considered an instance of neural posterior estimation (NPE), minimizes an expectation of the inclusive (forward) KL divergence to fit a variational distribution that is parameterized by a neural network. Our convergence result relies on the neural tangent kernel (NTK) to characterize the gradient dynamics that arise from considering the variational objective in function space. In the asymptotic regime of a fixed, positive-definite neural tangent kernel, we establish conditions under which the variational objective admits a unique solution in a reproducing kernel Hilbert space (RKHS). Then, we show that the gradient descent dynamics in function space converge to this unique function. In ablation studies and practical problems, we demonstrate that our results explain the behavior of NPE in non-asymptotic finite-neuron settings, and show that NPE outperforms ELBO-based optimization, which often converges to shallow local optima.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
