面向高维后验的隐式变分推断
Implicit Variational Inference for High-Dimensional Posteriors
AI总结：
该研究提出基于神经采样器局部线性化的隐式变分推断方法及新型采样器架构，可处理数千万潜变量的高维后验，能恢复大型贝叶斯神经网络跨层相关性，在下游任务上的不确定性量化效果优于现有最优方法。
摘要：
在变分推断中，贝叶斯模型的优势依赖于对真实后验分布的准确捕捉。我们提出使用定义隐式分布的神经采样器，这类采样器非常适合在高维空间中逼近复杂的多模态且具有相关性的后验。我们的方法通过对神经采样器进行局部线性化，提出了用于隐式分布近似推断的新型界，这区别于现有依赖额外判别器网络和不稳定对抗目标的方法。此外，我们提出了一种新的采样器架构，首次实现了数千万个潜变量上的隐式分布，并通过可微数值近似解决了计算问题。我们通过实验证明，该方法能够恢复大型贝叶斯神经网络（Bayesian neural network）跨层的相关性，这一性质对网络性能至关重要，但实现起来难度极高。据我们所知，尚无其他方法被证明能在如此大的模型上完成这一任务。通过下游任务实验，我们证明了我们的高表达力后验优于当前最优的不确定性量化方法，验证了我们训练算法的有效性和所学隐式近似的质量。
英文摘要：
In variational inference, the benefits of Bayesian models rely on accurately capturing the true posterior distribution. We propose using neural samplers that specify implicit distributions, which are well-suited for approximating complex multimodal and correlated posteriors in high-dimensional spaces. Our approach introduces novel bounds for approximate inference using implicit distributions by locally linearising the neural sampler. This is distinct from existing methods that rely on additional discriminator networks and unstable adversarial objectives. Furthermore, we present a new sampler architecture that, for the first time, enables implicit distributions over tens of millions of latent variables, addressing computational concerns by using differentiable numerical approximations. We empirically show that our method is capable of recovering correlations across layers in large Bayesian neural networks, a property that is crucial for a network's performance but notoriously challenging to achieve. To the best of our knowledge, no other method has been shown to accomplish this task for such large models. Through experiments in downstream tasks, we demonstrate that our expressive posteriors outperform state-of-the-art uncertainty quantification methods, validating the effectiveness of our training algorithm and the quality of the learned implicit approximation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
