基于Fisher散度得分匹配的高斯过程倾斜非参数密度估计
Gaussian Process Tilted Nonparametric Density Estimation using Fisher Divergence Score Matching
AI总结：
本文提出GP倾斜非参数密度估计器，基于Fisher散度得分匹配推导三种闭式学习算法，无需迭代优化仅需单次遍历数据，适配大规模数据集场景。
摘要：
我们提出了一种基于高斯过程（GP）的非参数密度估计器，并基于Fisher散度（FD）得分匹配推导了三种全新的闭式学习算法。该密度估计器通过将基础多元正态分布与指数化的高斯过程修正项相乘构建，因此我们将其称为GP倾斜非参数密度。通过使用随机傅里叶特征（RFF）近似将得分中的高斯过程部分表示为线性函数，我们证明所考虑的三种基于FD的目标函数均可通过闭式解完成优化，涵盖基础版Fisher散度、噪声条件版Fisher散度，以及本文提出的一种基于变分推断（VI）的噪声条件FD模型替代方案。针对这一全新学习方法，我们提出了类ELBO优化方法来近似后验分布，并据此推导得到Fisher变分预测分布。高斯过程的RFF表示在功能上等价于采用余弦激活函数的单层神经网络得分模型，为高斯过程提供了实用的线性表示，所有期望均可据此求解。高斯基础分布还有助于提升变分推断近似的易处理性，确保我们提出的密度是良定义的。我们在多个低维密度估计任务上验证了所提的三种学习算法以及MAP基线算法的效果。该学习问题的闭式特性使其无需依赖迭代学习算法，仅需单次遍历数据收集充分统计量即可完成计算，因此该技术尤其适配大规模数据集场景。
英文摘要：
We propose a nonparametric density estimator based on the Gaussian process (GP) and derive three novel closed form learning algorithms based on Fisher divergence (FD) score matching. The density estimator is formed by multiplying a base multivariate normal distribution with an exponentiated GP refinement, and so we refer to it as a GP-tilted nonparametric density. By representing the GP part of the score as a linear function using the random Fourier feature (RFF) approximation, we show that optimization can be solved in closed form for the three FD-based objectives considered. This includes the basic and noise conditional versions of the Fisher divergence, as well as an alternative to noise conditional FD models based on variational inference (VI) that we propose in this paper. For this novel learning approach, we propose an ELBO-like optimization to approximate the posterior distribution, with which we then derive a Fisher variational predictive distribution. The RFF representation of the GP, which is functionally equivalent to a single layer neural network score model with cosine activation, provides a useful linear representation of the GP for which all expectations can be solved. The Gaussian base distribution also helps with tractability of the VI approximation and ensures that our proposed density is well-defined. We demonstrate our three learning algorithms, as well as a MAP baseline algorithm, on several low dimensional density estimation problems. The closed form nature of the learning problem removes the reliance on iterative learning algorithms, making this technique particularly well-suited to big data sets, since only sufficient statistics collected from a single pass through the data is needed.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
