arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1909.02373stat.MLcs.LG
收藏

LSMI-Sinkhorn:基于最优传输的半监督互信息估计

LSMI-Sinkhorn: Semi-supervised Mutual Information Estimation with Optimal Transport

AAII, University of Technology Sydney(澳大利亚悉尼科技大学AAII);Kyoto University(京都大学);RIKEN AIP(理化学研究所人工智能项目);Carnegie Mellon University(卡内基梅隆大学)

Yanbin Liu, Makoto Yamada, Yao-Hung Hubert Tsai, Tam Le, Ruslan Salakhutdinov, Yi Yang

AI总结:

针对难以获取大量配对样本的互信息估计难题,提出基于最优传输的半监督LSMI-Sinkhorn算法,仅需少量配对样本结合非配对样本即可高效估计平方损失互信息,在图像匹配等任务中表现有效。

摘要:

互信息估计是统计学与机器学习领域的重要问题。从数据中估计互信息的常规做法是准备一组配对样本$\{(\boldsymbol{x}_i,\boldsymbol{y}_i)\\_\{i=1\}^n \stackrel{\mathrm{i.i.d.}}{\sim} p(\boldsymbol{x},\boldsymbol{y})\}$,但在很多场景下难以获取大量配对数据。针对这一问题,我们提出半监督平方损失互信息(Squared-loss Mutual Information, SMI)估计方法,仅需少量配对样本结合可用的非配对样本即可实现估计。我们首先通过密度比函数表示SMI,其中期望项通过边缘分布样本及其分配参数进行近似,目标函数基于最优传输问题与二次规划构建。随后,我们提出带Sinkhorn的最小二乘互信息(Least-Squares Mutual Information with Sinkhorn, LSMI-Sinkhorn)算法以实现高效优化。实验中,我们首先验证了所提方法无需大量配对样本即可估计SMI,进一步证明了LSMI-Sinkhorn算法在图像匹配、相册摘要生成等各类机器学习问题上的有效性。代码可在https://github.com/csyanbin/LSMI-Sinkhorn获取。

英文摘要:

Estimating mutual information is an important statistics and machine learning problem. To estimate the mutual information from data, a common practice is preparing a set of paired samples $\{(\mathbf{x}_i,\mathbf{y}_i)\}_{i=1}^n \stackrel{\mathrm{i.i.d.}}{\sim} p(\mathbf{x},\mathbf{y})$. However, in many situations, it is difficult to obtain a large number of data pairs. To address this problem, we propose the semi-supervised Squared-loss Mutual Information (SMI) estimation method using a small number of paired samples and the available unpaired ones. We first represent SMI through the density ratio function, where the expectation is approximated by the samples from marginals and its assignment parameters. The objective is formulated using the optimal transport problem and quadratic programming. Then, we introduce the Least-Squares Mutual Information with Sinkhorn (LSMI-Sinkhorn) algorithm for efficient optimization. Through experiments, we first demonstrate that the proposed method can estimate the SMI without a large number of paired samples. Then, we show the effectiveness of the proposed LSMI-Sinkhorn algorithm on various types of machine learning problems such as image matching and photo album summarization. Code can be found at https://github.com/csyanbin/LSMI-Sinkhorn.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑