降低用于MCMC预处理的归一化流复杂度
Reducing normalizing flow complexity for MCMC preconditioning
AI总结：
提出一种结合线性预处理器与条件归一化流的分解预处理架构，降低模型复杂度并提升对目标几何的适应性，在多种复杂分布和贝叶斯模型上显著提升了采样效率与尾部样本质量。
摘要：
预处理是MCMC算法的关键组件，通过一个可逆映射促进对几何复杂目标分布的探索，从而提高采样效率。线性预处理器通常足以应对中等复杂的目标分布，而最近的研究探索了使用可逆神经网络作为归一化流（NF）组件的非线性预处理。然而，实证和理论研究表明，过度参数化的NF预处理器会降低采样效率和拟合质量。此外，现有的基于NF的方法不会根据目标分布调整其架构。MCMC之外的相关研究同样发现，适当参数化的NF可以用更少的训练时间或数据实现相当或更优的性能。我们提出了一种分解预处理架构，通过将线性组件与条件NF相结合来降低NF复杂度，从而提高对目标几何的适应性。线性预处理器应用于从预热样本估计出的近似高斯分布的维度，而条件NF则对更复杂的维度进行建模。我们的方法在两个复杂的合成分布上产生了显著更好的尾部样本，并在具有不同似然和先验强度的稀疏逻辑回归后验上表现出持续更好的性能。它还在具有弱似然和强漏斗几何的分层贝叶斯模型后验上实现了更高的有效样本量。这种方法与数据有限的分层贝叶斯模型分析尤为相关，并可能为当前神经MCMC设计在理论和软件方面的进展提供参考。
英文摘要：
Preconditioning is a key component of MCMC algorithms that improves sampling efficiency by facilitating exploration of geometrically complex target distributions through an invertible map. While linear preconditioners are often sufficient for moderately complex target distributions, recent work has explored nonlinear preconditioning with invertible neural networks as components of normalizing flows (NFs). However, empirical and theoretical studies show that overparameterized NF preconditioners can degrade sampling efficiency and fit quality. Moreover, existing NF-based approaches do not adapt their architectures to the target distribution. Related work outside of MCMC similarly finds that suitably parameterized NFs can achieve comparable or superior performance with substantially less training time or data. We propose a factorized preconditioning architecture that reduces NF complexity by combining a linear component with a conditional NF, improving adaptability to target geometry. The linear preconditioner is applied to dimensions that are approximately Gaussian, as estimated from warmup samples, while the conditional NF models more complex dimensions. Our method yields significantly better tail samples on two complex synthetic distributions and consistently better performance on a sparse logistic regression posterior across varying likelihood and prior strengths. It also achieves higher effective sample sizes on hierarchical Bayesian model posteriors with weak likelihoods and strong funnel geometries. This approach is particularly relevant for hierarchical Bayesian model analyses with limited data and could inform current theoretical and software strides in neural MCMC design.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
