arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2305.16841cs.LGcs.AI
收藏

可微随机划分模型

Differentiable Random Partition Models

ETH Zurich(苏黎世联邦理工学院)

Thomas M. Sutter, Alain Ryser, Joram Liebeskind, Julia E. Vogt

AI总结:

针对划分问题的不可微性阻碍端到端优化的问题,提出一种两步可微随机划分模型,可计算重参数化梯度,在变分聚类等三类任务中验证了通用性。

摘要:

将一组元素划分为数量未知的互斥子集在诸多机器学习问题中至关重要。然而,将元素(例如数据集中的样本或网络层中的神经元)分配到数量未知的离散子集本质上是不可微的,这阻碍了基于梯度的端到端参数优化。我们通过提出一种用于推断划分的新型两步法克服了这一限制,使其可应用于变分推断任务。该新方法能够计算关于新型随机划分模型参数的重参数化梯度。我们的方法首先推断每个子集的元素数量,其次按照学习到的顺序填充这些子集。我们在三项不同的挑战性实验中展示了这一通用方法的多功能性:变分聚类、弱监督下共享与独立生成因子的推断,以及多任务学习。

英文摘要:

Partitioning a set of elements into an unknown number of mutually exclusive subsets is essential in many machine learning problems. However, assigning elements, such as samples in a dataset or neurons in a network layer, to an unknown and discrete number of subsets is inherently non-differentiable, prohibiting end-to-end gradient-based optimization of parameters. We overcome this limitation by proposing a novel two-step method for inferring partitions, which allows its usage in variational inference tasks. This new approach enables reparameterized gradients with respect to the parameters of the new random partition model. Our method works by inferring the number of elements per subset and, second, by filling these subsets in a learned order. We highlight the versatility of our general-purpose approach on three different challenging experiments: variational clustering, inference of shared and independent generative factors under weak supervision, and multitask learning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑