arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2006.12204stat.MLcs.LG
收藏

伸缩密度比估计

Telescoping Density-Ratio Estimation

School of Informatics(信息学院);University of Edinburgh(爱丁堡大学)

Benjamin Rhodes, Kai Xu, Michael U. Gutmann

摘要:

通过分类进行密度比估计是无监督学习的基石。它为表示学习和生成建模的最先进方法提供了基础,用例的数量继续激增。然而,它有一个关键的限制:它不能准确地估计两个密度显著不同的p/q比。根据经验,我们发现,每当p和q之间的库尔贝克-莱布勒散度超过数十nat时,就会发生这种情况。为了解决这个问题,我们引入了一个新的框架,伸缩密度比估计(TRE),使高度不相似的密度在高维空间之间的比率估计。我们的实验表明,TRE可以产生实质性的改进,现有的单比方法的互信息估计,表示学习和基于能量的建模。

英文摘要:

Density-ratio estimation via classification is a cornerstone of unsupervised learning. It has provided the foundation for state-of-the-art methods in representation learning and generative modelling, with the number of use-cases continuing to proliferate. However, it suffers from a critical limitation: it fails to accurately estimate ratios p/q for which the two densities differ significantly. Empirically, we find this occurs whenever the KL divergence between p and q exceeds tens of nats. To resolve this limitation, we introduce a new framework, telescoping density-ratio estimation (TRE), that enables the estimation of ratios between highly dissimilar densities in high-dimensional spaces. Our experiments demonstrate that TRE can yield substantial improvements over existing single-ratio methods for mutual information estimation, representation learning and energy-based modelling.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑