多速率Stein变分梯度下降用于高效的贝叶斯采样
Multirate Stein Variational Gradient Descent for Efficient Bayesian Sampling
AI总结：
本文提出多速率SVGD方法，通过不同时间尺度更新不同组件，提升高维、各向异性或分层后验中的鲁棒性和效率，尤其在刚性分层、强各向异性及多模态目标中表现最佳。
摘要：
许多基于粒子的贝叶斯推断方法使用单一全局步长进行所有更新。然而，在Stein变分梯度下降(SVGD)中，每个更新结合了两种不同效应：吸引高后验区域和保持粒子多样性的排斥。这些效应可以以不同速率演变，尤其是在高维、各向异性或分层后验中，因此单一步长在某些区域不稳定，其他区域效率低下。我们推导了SVGD的多速率版本，以不同时间尺度更新这些组件。该框架产生实用算法，包括对称分裂法、固定多速率法(MR-SVGD)和具有局部误差控制的自适应多速率法(Adapt-MR-SVGD)。我们评估了这些方法在广泛且严谨的基准套件中，涵盖六个问题家族：50维高斯目标、多个2D合成目标、UCI贝叶斯逻辑回归、多模态高斯混合、贝叶斯神经网络和大规模分层逻辑回归。评估包括后验匹配度量、预测性能、校准质量、混合和显式计算成本。在这些六个基准家族中，多速率SVGD变体在鲁棒性和质量-成本权衡方面优于原生SVGD。最强的收益出现在刚性分层、强各向异性及多模态目标中，其中自适应多速率SVGD通常是最佳变体，而固定多速率SVGD提供一个更简单的鲁棒替代方案，成本更低。
英文摘要：
Many particle-based Bayesian inference methods use a single global step size for all parts of the update. In Stein variational gradient descent (SVGD), however, each update combines two qualitatively different effects: attraction toward high-posterior regions and repulsion that preserves particle diversity. These effects can evolve at different rates, especially in high-dimensional, anisotropic, or hierarchical posteriors, so one step size can be unstable in some regions and inefficient in others. We derive a multirate version of SVGD that updates these components on different time scales. The framework yields practical algorithms, including a symmetric split method, a fixed multirate method (MR-SVGD), and an adaptive multirate method (Adapt-MR-SVGD) with local error control. We evaluate the methods in a broad and rigorous benchmark suite covering six problem families: a 50D Gaussian target, multiple 2D synthetic targets, UCI Bayesian logistic regression, multimodal Gaussian mixtures, Bayesian neural networks, and large-scale hierarchical logistic regression. Evaluation includes posterior-matching metrics, predictive performance, calibration quality, mixing, and explicit computational cost accounting. Across these six benchmark families, multirate SVGD variants improve robustness and quality-cost tradeoffs relative to vanilla SVGD. The strongest gains appear on stiff hierarchical, strongly anisotropic, and multimodal targets, where adaptive multirate SVGD is usually the strongest variant and fixed multirate SVGD provides a simpler robust alternative at lower cost.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
