arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2402.17287cs.LGcs.CVstat.ML
收藏

对生成模型基于熵的新颖性的可解释评估

An Interpretable Evaluation of Entropy-based Novelty of Generative Models

The Chinese University of Hong Kong(香港中文大学)

Jingwei Zhang, Cheuk Ting Li, Farzan Farnia

AI总结:

针对生成模型相对参考模型的新颖性评估不足,本文提出谱差分聚类方法与 KEN 分数,量化多模态分布中的新模态,并在合成及真实图像数据上验证有效性。

摘要:

生成模型框架的大规模发展要求采用有原则的方法来评估模型相对于参考数据集的新颖性。尽管文献已广泛研究了生成模型的质量、多样性和泛化能力评估,但在机器学习领域,模型相对于参考模型的新颖性评估尚未得到充分探索。在本工作中,我们聚焦多模态分布的新颖性评估,并尝试解决以下差分聚类任务:给定生成模型 $P_\mathcal{G}$ 和参考模型 $P_\mathrm{ref}$ 的样本,我们如何发现 $P_\mathcal{G}$ 比 $P_\mathrm{ref}$ 更频繁表达的样本类型?我们为差分聚类任务引入一种谱方法,并提出基于核的熵新颖性(Kernel-based Entropic Novelty,KEN)分数,以量化 $P_\mathcal{G}$ 相对于 $P_\mathrm{ref}$ 的基于模态的新颖性。我们分析了具有良好可分成分的混合分布的 KEN 分数,并开发了一种基于核的方法来从经验数据计算 KEN 分数。我们通过在合成和真实图像数据集上的数值结果为 KEN 框架提供支持,结果表明该框架在检测新颖模态和比较生成模型方面的有效性。论文代码可在 www.github.com/buyeah1109/KEN 获取。

英文摘要:

The massive developments of generative model frameworks require principled methods for the evaluation of a model's novelty compared to a reference dataset. While the literature has extensively studied the evaluation of the quality, diversity, and generalizability of generative models, the assessment of a model's novelty compared to a reference model has not been adequately explored in the machine learning community. In this work, we focus on the novelty assessment for multi-modal distributions and attempt to address the following differential clustering task: Given samples of a generative model $P_\mathcal{G}$ and a reference model $P_\mathrm{ref}$, how can we discover the sample types expressed by $P_\mathcal{G}$ more frequently than in $P_\mathrm{ref}$? We introduce a spectral approach to the differential clustering task and propose the Kernel-based Entropic Novelty (KEN) score to quantify the mode-based novelty of $P_\mathcal{G}$ with respect to $P_\mathrm{ref}$. We analyze the KEN score for mixture distributions with well-separable components and develop a kernel-based method to compute the KEN score from empirical data. We support the KEN framework by presenting numerical results on synthetic and real image datasets, indicating the framework's effectiveness in detecting novel modes and comparing generative models. The paper's code is available at: www.github.com/buyeah1109/KEN

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑