arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1905.11112stat.MLcs.ITcs.LGmath.IT
收藏

f-散度的实用相合估计

Practical and Consistent Estimation of f-Divergences

Max Planck Institute for Intelligent Systems(马克斯·普朗克智能系统研究所);University of Cambridge(剑桥大学);Google Research(谷歌研究院)

Paul K. Rubenstein, Olivier Bousquet, Josip Djolonga, Carlos Riquelme, Ilya Tolstikhin

摘要:

基于样本的两个概率分布之间的f-散度的估计是统计学和机器学习中的基本问题。大多数作品在非常弱的假设下研究这个问题,在这种情况下,它是可证明的困难。我们考虑了现代机器学习中通常满足的更强结构假设的情况,包括表示学习和自动编码器架构的生成建模。在这些假设下,我们提出并研究了一个估计,可以很容易地实现,在高维工作良好,并享有更快的收敛速度。我们验证我们的估计经验的行为在合成和真实数据的实验,并讨论其直接影响总相关性,熵和互信息估计。

英文摘要:

The estimation of an f-divergence between two probability distributions based on samples is a fundamental problem in statistics and machine learning. Most works study this problem under very weak assumptions, in which case it is provably hard. We consider the case of stronger structural assumptions that are commonly satisfied in modern machine learning, including representation learning and generative modelling with autoencoder architectures. Under these assumptions we propose and study an estimator that can be easily implemented, works well in high dimensions, and enjoys faster rates of convergence. We verify the behavior of our estimator empirically in both synthetic and real-data experiments, and discuss its direct implications for total correlation, entropy, and mutual information estimation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑