arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2111.00875cs.LGcs.PFstat.MEstat.ML
收藏

一种用于潜变量生成模型的矩匹配度量

A moment-matching metric for latent variable generative models

Université du Québec à Montréal(魁北克大学蒙特利尔分校)

Cédric Beaulac

AI总结:

针对无监督学习中潜变量生成模型似然度量失效的问题,提出基于Frobenius范数等矩阵范数的矩匹配度量,用于模型比较与正则化,相较采样评估计算更快、方差更小,并给出概念验证。

摘要:

在无监督学习问题中,评估拟合模型的质量往往十分困难。潜变量模型(如变分自编码器variation autoencoders、高斯混合模型Gaussian mixture models)通常采用基于似然的方法进行训练。在古德哈特定律(Goodhart's law)的范畴下,当一项度量成为目标时,它就不再是一个好的度量,因此我们不应使用似然来评估这些模型的拟合质量。我们提出的解决方案是一种基于矩的新型度量,可用于模型比较或正则化。其核心思路是利用矩阵范数(如Frobenius范数)研究数据矩与模型矩之间的差异。我们展示了如何将这一新度量用于模型比较,随后又将其用于正则化。评估潜变量模型时,从拟合分布中采样是常见做法,而我们表明,所提出的度量计算速度更快,且方差比该替代方法更小。本文最后给出了两种应用的概念验证,并对未来工作进行了讨论。

英文摘要:

It can be difficult to assess the quality of a fitted model when facing unsupervised learning problems. Latent variable models, such as variation autoencoders and Gaussian mixture models, are often trained with likelihood-based approaches. In scope of Goodhart's law, when a metric becomes a target it ceases to be a good metric and therefore we should not use likelihood to assess the quality of the fit of these models. The solution we propose is a new metric for model comparison or regularization that relies on moments. The concept is to study the difference between the data moments and the model moments using a matrix norm, such as the Frobenius norm. We show how to use this new metric for model comparison and then for regularization. It is common to draw samples from the fitted distribution when evaluating latent variable models and we show that our proposed metric is faster to compute and has a smaller variance that this alternative. We conclude this article with a proof of concept of both applications and we discuss future work.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑