arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1302.5125stat.MLcs.LG
收藏

深密度模型下的高维概率估计

High-Dimensional Probability Estimation with Deep Density Models

Massachusetts Institute of Technology(麻省理工学院);Harvard University(哈佛大学)

Oren Rippel, Ryan Prescott Adams

摘要:

机器学习中的一个基本问题是从数据中估计概率分布。已经提出了许多技术来研究数据的结构,大多数情况下都是围绕着这样一个假设,即观测值位于高概率的低维流形上。然而,利用这种洞察力为高维数据构建明确的,易于处理的密度模型更加困难。本文介绍了一种新的密度估计方法--深度密度模型(DDM)。我们利用深度学习的见解来构建一个到表示空间的双射映射,在该映射下,数据分布的变换被近似分解,并且具有相同的已知边缘密度。该模型下的潜在分布的简单性使我们能够可行地探索它,并且映射的可逆性可以表征测量的收缩。这使我们能够计算样本外数据的归一化密度。这种易处理性和灵活性的结合使我们能够处理高维数据集上的各种概率任务,包括:在测试时快速计算归一化密度,而无需评估分区函数;在没有MCMC的情况下生成样本;以及表征数据的联合熵。

英文摘要:

One of the fundamental problems in machine learning is the estimation of a probability distribution from data. Many techniques have been proposed to study the structure of data, most often building around the assumption that observations lie on a lower-dimensional manifold of high probability. It has been more difficult, however, to exploit this insight to build explicit, tractable density models for high-dimensional data. In this paper, we introduce the deep density model (DDM), a new approach to density estimation. We exploit insights from deep learning to construct a bijective map to a representation space, under which the transformation of the distribution of the data is approximately factorized and has identical and known marginal densities. The simplicity of the latent distribution under the model allows us to feasibly explore it, and the invertibility of the map to characterize contraction of measure across it. This enables us to compute normalized densities for out-of-sample data. This combination of tractability and flexibility allows us to tackle a variety of probabilistic tasks on high-dimensional datasets, including: rapid computation of normalized densities at test-time without evaluating a partition function; generation of samples without MCMC; and characterization of the joint entropy of the data.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑