arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.04355stat.MLcs.AIcs.LGstat.ME
收藏

PQMass:基于概率质量估计的生成模型质量概率评估

PQMass: Probabilistic Assessment of the Quality of Generative Models using Probability Mass Estimation

Mila – Québec Artificial Intelligence Institute(魁北克人工智能研究所);Ciela Institute(Ciela研究所);Université de Montréal(蒙特利尔大学);SanboxAQ;CCA – Flatiron Institute(CCA-Flatiron研究所);University of Edinburgh(爱丁堡大学);Trottier Space Institute(特罗特里尔空间研究所)

Pablo Lemos, Sammy Sharief, Esmeralda S. Whitammer, Salma Salhi, Connor Stone, Laurence Perreault-Levasseur, Yashar Hezaveh

AI总结:

研究提出无需似然的PQMass方法,通过划分样本空间并应用卡方检验,可严谨评估生成模型质量、比较模型,适用于多模态及中等高维数据,无需辅助模型或特征提取。

摘要:

我们提出了一种无需似然的方法,用于在给定来自两个分布的样本时对其进行比较,目标是评估生成模型的质量。所提出的方法PQMass提供了一种统计上严谨的方法,可用于评估单个生成模型的性能或比较多个竞争模型。PQMass将样本空间划分为不重叠的区域,并对落在每个区域内的数据样本数量应用卡方检验,得到一个p值,该p值用于衡量来自两组样本的分箱计数是否来自同一多项分布的概率。PQMass不依赖于关于真实分布密度的假设,也不依赖于训练或拟合任何辅助模型。我们在不同模态和维度的数据上评估了PQMass,证明其在评估生成样本的质量、新颖性和多样性方面的有效性。我们进一步表明,PQMass可良好扩展至中等高维数据,因此在实际应用中无需进行特征提取。

英文摘要:

We propose a likelihood-free method for comparing two distributions given samples from each, with the goal of assessing the quality of generative models. The proposed approach, PQMass, provides a statistically rigorous method for assessing the performance of a single generative model or the comparison of multiple competing models. PQMass divides the sample space into non-overlapping regions and applies chi-squared tests to the number of data samples that fall within each region, giving a p-value that measures the probability that the bin counts derived from two sets of samples are drawn from the same multinomial distribution. PQMass does not depend on assumptions regarding the density of the true distribution, nor does it rely on training or fitting any auxiliary models. We evaluate PQMass on data of various modalities and dimensions, demonstrating its effectiveness in assessing the quality, novelty, and diversity of generated samples. We further show that PQMass scales well to moderately high-dimensional data and thus obviates the need for feature extraction in practical applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑