提出 Vendi Score 多样性评估指标，基于相似矩阵特征值的 Shannon 熵，无需参考数据集，适用于任意生成模型与数据集，揭示了 GAN 模式崩溃等问题并支持数据增强。。
摘要：
多样性是机器学习（ML）许多领域的重要标准，包括生成式建模和数据集筛选。然而，现有的衡量多样性的指标通常是特定领域的，且灵活性有限。在本文中，我们通过提出 Vendi Score 来解决多样性评估问题，它将生态学和量子统计力学中的思想与 ML 联系并扩展起来。Vendi Score 被定义为相似矩阵特征值的 Shannon 熵的指数。该矩阵由应用于待评估多样性样本的用户定义相似性函数导出。通过将相似性函数作为输入，Vendi Score 使其用户能够指定任何所需的多样性形式。重要的是，与 ML 中许多现有指标不同，Vendi Score 不需要参考数据集或样本或标签上的分布，因此它是通用的，适用于任何生成模型、解码算法和任何可以定义相似性的领域的数据集。我们在分子生成式建模中展示了 Vendi Score，发现它解决了该领域当前首选多样性指标的缺点。我们还将 Vendi Score 应用于图像生成模型和文本解码算法，发现它证实了这些领域关于多样性的已知结果。此外，我们使用 Vendi Score 衡量了模式崩溃（mode collapse），这是生成对抗网络（GANs）的一个已知缺点。具体而言，Vendi Score 揭示了即使捕获了带标签数据集所有模式的 GANs，其多样性也可能低于原始数据集。最后，Vendi Score 的可解释性使我们能够诊断几个基准 ML 数据集的多样性，为多样性知情的數据增强打开了大门。
英文摘要：
Diversity is an important criterion for many areas of machine learning (ML), including generative modeling and dataset curation. However, existing metrics for measuring diversity are often domain-specific and limited in flexibility. In this paper, we address the diversity evaluation problem by proposing the Vendi Score, which connects and extends ideas from ecology and quantum statistical mechanics to ML. The Vendi Score is defined as the exponential of the Shannon entropy of the eigenvalues of a similarity matrix. This matrix is induced by a user-defined similarity function applied to the sample to be evaluated for diversity. In taking a similarity function as input, the Vendi Score enables its user to specify any desired form of diversity. Importantly, unlike many existing metrics in ML, the Vendi Score does not require a reference dataset or distribution over samples or labels, it is therefore general and applicable to any generative model, decoding algorithm, and dataset from any domain where similarity can be defined. We showcase the Vendi Score on molecular generative modeling where we found it addresses shortcomings of the current diversity metric of choice in that domain. We also applied the Vendi Score to generative models of images and decoding algorithms of text where we found it confirms known results about diversity in those domains. Furthermore, we used the Vendi Score to measure mode collapse, a known shortcoming of generative adversarial networks (GANs). In particular, the Vendi Score revealed that even GANs that capture all the modes of a labeled dataset can be less diverse than the original dataset. Finally, the interpretability of the Vendi Score allowed us to diagnose several benchmark ML datasets for diversity, opening the door for diversity-informed data augmentation.
Conditional Vendi Score: Prompt-Aware Diversity Evaluation for Generative AI Models and LLMs
Department of Computer Science and Engineering, The Chinese University of Hong Kong（计算机科学与工程系，香港中文大学）；Department of Information Engineering, The Chinese University of Hong Kong（信息工程系，香港中文大学）
针对文本提示引导的生成模型，提出条件 Vendi 和条件 RKE 分数，通过条件熵分离模型自身多样性，并证明收敛性及在多个任务中恢复真实多样性排序。
How Much Is a Dataset Worth? Scaling Laws, the Vendi Score, and Matrix Spectral Functions
Department of Electrical & Computer Engineering（电气与计算机工程系）；Paul G. Allen School of Computer Science & Engineering（保罗·G·艾伦计算机科学与工程学院）；University of Washington（华盛顿大学）
GM Score: Incorporating inter-class and intra-class generator diversity, discriminability of disentangled representation, and sample fidelity for evaluating GANs
School of Computer Engineering, KIIT Deemed to be University（KIIT 大学计算机工程学院）
Vendi Novelty Scores for Out-of-Distribution Detection
Lewis-Sigler Institute For Integrative Genomics, Princeton University（普林斯顿大学整合基因组学研究所）；Department of Computer Science, Princeton University（普林斯顿大学计算机科学系）
SPARKE: Scalable Prompt-Aware Diversity and Novelty Guidance in Diffusion Models via RKE Score
Department of Computer Science and Engineering, The Chinese University of Hong Kong（计算机科学与工程系，香港中文大学）；Department of Information Engineering, The Chinese University of Hong Kong（信息工程系，香港中文大学）
The Alpha-Alternator: Dynamic Adaptation To Varying Noise Levels In Sequences Using The Vendi Score For Improved Robustness and Performance
Institute of Biomedical Engineering, University of Toronto（生物医学工程研究所，多伦多大学）；Department of Computer Science, Princeton University（计算机科学系，普林斯顿大学）；Vertaix
Debias Anything: Fairness with Diversity without Supervision in Diffusion Models
Criteo AI Lab（Criteo AI实验室）；ESSEC Business School（ESSEC商学院）；CentraleSupélec, Université Paris-Saclay（巴黎萨克雷大学中央理工-高等电力学院）；FairPlay joint team（FairPlay联合团队）
Flow Score Distillation for Diverse Text-to-3D Generation
Institute for Interdisciplinary Information Sciences, Tsinghua University（清华大学交叉信息研究院）；ArchipLab, Institute for Interdisciplinary Information Sciences, Tsinghua University（清华大学交叉信息研究院 ArchipLab）
ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation
Tsinghua-Bosch Joint ML Center（清华-博世联合机器学习中心）；ShengShu；Pazhou Laboratory (Huangpu)（琶洲实验室（黄埔））；Beijing Key Laboratory of Big Data Management and Analysis Methods（大数据管理与分析方法北京市重点实验室）
