arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1903.10567cs.LGstat.ML
收藏

广义概率面优化与对数密度估计

General Probabilistic Surface Optimization and Log Density Estimation

Technion - Israel Institute of Technology(以色列理工学院)

Dmitry Kopitkov, Vadim Indelman

摘要:

在本文中,我们贡献了一个新的算法家族,它概括了许多无监督的技术,包括非归一化和能量模型,并允许我们从数据样本中推断不同的统计模式(例如数据似然和密度之间的比率)。所提出的无监督技术称为概率表面优化(PSO),将模型视为柔性表面,可以根据特定于损失的虚拟随机力来推动,当表面上的逐点力变得相等时,实现动态平衡。具体地说,表面在从两个不同分布采样的点处被上下推动。平均向上和向下的力成为这两个分布密度和由特定PSO实例的损失定义的力大小的函数。在收敛时,力平衡根据所使用的量值函数将优化模型施加为等于各种统计函数。此外,这种动态统计平衡是非常直观和有用的,提供了许多含义和概率推理的可能用途。我们将PSO连接到许多现有的统计工作,这也是PSO的实例,并推导出新的基于PSO的推理方法,证明PSO的特殊可用性。同样,基于来自虚拟力的角度来看,我们分析PSO的稳定性,并提出新的方法来提高它。最后,我们提出了新的PSO实例,称为PSO-LDE,数据对数密度估计,也提供了一个新的NN块对角架构,增加表面的灵活性,这显着提高了估计精度。这两个PSO-LDE和新的架构结合在一起,作为一种新的密度估计技术。在我们的实验中,我们证明了这种技术优于国家的最先进的基线在密度估计任务的多模态20 D数据。

英文摘要:

In this paper we contribute a novel algorithm family, which generalizes many unsupervised techniques including unnormalized and energy models, and allows us to infer different statistical modalities (e.g. data likelihood and ratio between densities) from data samples. The proposed unsupervised technique, named Probabilistic Surface Optimization (PSO), views a model as a flexible surface which can be pushed according to loss-specific virtual stochastic forces, where a dynamical equilibrium is achieved when the pointwise forces on the surface become equal. Concretely, the surface is pushed up and down at points sampled from two different distributions. The averaged up and down forces become functions of these two distribution densities and of force magnitudes defined by the loss of a particular PSO instance. Upon convergence, the force equilibrium imposes an optimized model to be equal to various statistical functions depending on the used magnitude functions. Furthermore, this dynamical-statistical equilibrium is extremely intuitive and useful, providing many implications and possible usages in probabilistic inference. We connect PSO to numerous existing statistical works which are also PSO instances, and derive new PSO-based inference methods as demonstration of PSO exceptional usability. Likewise, based on the insights coming from the virtual-force perspective we analyze PSO stability and propose new ways to improve it. Finally, we present new instances of PSO, termed PSO-LDE, for data log-density estimation and also provide a new NN block-diagonal architecture for increased surface flexibility, which significantly improves estimation accuracy. Both PSO-LDE and the new architecture are combined together as a new density estimation technique. In our experiments we demonstrate this technique to be superior over state-of-the-art baselines in density estimation task for multimodal 20D data.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑