学习组重要性:基于可微超几何分布
Learning Group Importance using the Differentiable Hypergeometric Distribution
AI总结:
本文提出可微超几何分布,通过可重参数化梯度学习组大小,在弱监督学习和聚类中优于现有启发式方法。
摘要:
在许多应用中,将一组元素划分为大小先验未知的子集是至关重要的。这些子集大小很少被显式地学习——无论是聚类应用中的簇大小,还是弱监督学习中共享与独立生成潜因子的数量。由于硬约束,正确子集大小组合上的概率分布是不可微的,这阻碍了基于梯度的优化。在这项工作中,我们提出了可微超几何分布。超几何分布基于各组相对重要性来建模不同组大小的概率。我们引入了可重参数化的梯度来学习组间的重要性,并强调了在两个典型应用(弱监督学习和聚类)中显式学习子集大小的优势。在这两个应用中,我们都优于以往依赖次优启发式方法来建模未知组大小的方法。
英文摘要:
Partitioning a set of elements into subsets of a priori unknown sizes is essential in many applications. These subset sizes are rarely explicitly learned - be it the cluster sizes in clustering applications or the number of shared versus independent generative latent factors in weakly-supervised learning. Probability distributions over correct combinations of subset sizes are non-differentiable due to hard constraints, which prohibit gradient-based optimization. In this work, we propose the differentiable hypergeometric distribution. The hypergeometric distribution models the probability of different group sizes based on their relative importance. We introduce reparameterizable gradients to learn the importance between groups and highlight the advantage of explicitly learning the size of subsets in two typical applications: weakly-supervised learning and clustering. In both applications, we outperform previous approaches, which rely on suboptimal heuristics to model the unknown size of groups.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。