arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2110.02787stat.MLcs.LGmath.STstat.TH
收藏

非标准化分布的相对熵梯度采样器

Relative Entropy Gradient Sampler for Unnormalized Distributions

Shanghai University of Finance and Economics(上海财经大学);University of Iowa(爱荷华大学);Wuhan University(武汉大学)

Xingdong Feng, Yuan Gao, Jian Huang, Yuling Jiao, Xu Liu

摘要:

我们提出了一个相对熵梯度采样器(REGS)的采样从非归一化分布。REGS是一种粒子方法,它寻求一系列简单的非线性变换,迭代地将初始样本从参考分布推到来自未归一化目标分布的样本中。为了确定每次迭代的非线性变换,我们考虑了相对熵的Wasserstein梯度流。该梯度流确定内插参考分布和目标分布的概率分布的路径。它的特点是一个常微分方程系统的速度场取决于密度比的发展粒子的密度和非归一化的目标密度。为了使用REGS进行采样,我们需要估计密度比并模拟具有粒子演化的ODE系统。我们提出了一种新的非参数方法估计对数密度比使用神经网络。对具有挑战性的多模态1D和2D混合分布和贝叶斯逻辑斯谛回归在真实数据集上的广泛模拟研究表明,REGS优于比较中包含的最先进的采样方法。

英文摘要:

We propose a relative entropy gradient sampler (REGS) for sampling from unnormalized distributions. REGS is a particle method that seeks a sequence of simple nonlinear transforms iteratively pushing the initial samples from a reference distribution into the samples from an unnormalized target distribution. To determine the nonlinear transforms at each iteration, we consider the Wasserstein gradient flow of relative entropy. This gradient flow determines a path of probability distributions that interpolates the reference distribution and the target distribution. It is characterized by an ODE system with velocity fields depending on the density ratios of the density of evolving particles and the unnormalized target density. To sample with REGS, we need to estimate the density ratios and simulate the ODE system with particle evolution. We propose a novel nonparametric approach to estimating the logarithmic density ratio using neural networks. Extensive simulation studies on challenging multimodal 1D and 2D mixture distributions and Bayesian logistic regression on real datasets demonstrate that the REGS outperforms the state-of-the-art sampling methods included in the comparison.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑