arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2112.03440cs.LGstat.ML
收藏

多分布密度比估计的统一框架

A Unified Framework for Multi-distribution Density Ratio Estimation

Stanford University(斯坦福大学)

Lantao Yu, Yujia Jin, Stefano Ermon

摘要:

二进制密度比估计(DRE),即在给定经验样本的情况下估计$p_1/p_2$的比率的问题,为许多最先进的机器学习算法提供了基础,例如对比表示学习和协变量偏移自适应。在这项工作中,我们考虑一个广义的设置,其中给定的样本从多个分布$p_1, \ldots, p_k$(为$k > 2$),我们的目标是有效地估计所有对分布之间的密度比。这样的推广导致了重要的新应用,如估计多个随机变量之间的统计差异,如多分布$f$ -发散,并通过多个重要性采样的偏差校正。然后,我们从Bregman发散最小化的角度开发了一个一般框架,其中每个严格凸的多元函数引起多分布DRE的适当损失。此外,我们重新推导了多分布密度比估计和类概率估计之间的理论联系,证明了使用任何严格适当的评分规则与多分布DRE的链接函数的组合。我们表明,我们的框架导致的方法,严格概括其对应的二进制DRE,以及新的方法,显示出相当或优越的性能在各种下游任务。

英文摘要:

Binary density ratio estimation (DRE), the problem of estimating the ratio $p_1/p_2$ given their empirical samples, provides the foundation for many state-of-the-art machine learning algorithms such as contrastive representation learning and covariate shift adaptation. In this work, we consider a generalized setting where given samples from multiple distributions $p_1, \ldots, p_k$ (for $k > 2$), we aim to efficiently estimate the density ratios between all pairs of distributions. Such a generalization leads to important new applications such as estimating statistical discrepancy among multiple random variables like multi-distribution $f$-divergence, and bias correction via multiple importance sampling. We then develop a general framework from the perspective of Bregman divergence minimization, where each strictly convex multivariate function induces a proper loss for multi-distribution DRE. Moreover, we rederive the theoretical connection between multi-distribution density ratio estimation and class probability estimation, justifying the use of any strictly proper scoring rule composite with a link function for multi-distribution DRE. We show that our framework leads to methods that strictly generalize their counterparts in binary DRE, as well as new methods that show comparable or superior performance on various downstream tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑