PLDiv：基于持续态景观的数据集多样性与语义崩溃度量方法
PLDiv: Measuring Dataset Diversity and Semantic Collapse via Persistence Landscapes
AI总结：
本文提出PLDiv，一种基于持续态景观的多样性度量，无需特定任务监督即可量化数据集结构组织，在测量文本生成等的内在多样性、检测语义崩溃时表现优于基线，可支持合成数据监控等应用。
摘要：
现代数据管道越来越多地将人工创建、人工整理的数据与AI生成的数据结合起来，用于模型训练和评估。随着数据集规模增长，它们往往会添加语义冗余样本，而非扩展语义覆盖范围，这会降低整个数据集的质量。现有的多样性度量主要总结词汇变异、熵或成对相似度，但对底层数据分布的形态刻画有限。本文提出PLDiv，一种源自持续态景观的多样性度量，它从数据集的几何表示中量化其结构组织。PLDiv总结了随着附近样本逐步连接，数据邻域如何演化，生成一种可解释的结构多样性标量度量，无需特定任务监督。无论是测量文本生成或图像嵌入的内在多样性，还是检测指令数据集和推理轨迹中的语义崩溃，PLDiv的表现均优于所评估的基线，且与语义崩溃期间的多样性标签高度一致。因此，它能有效支持合成数据监控、数据集构建、数据增强和生成模型评估等应用。
英文摘要：
Modern data pipelines increasingly combine human-created and human-curated data with AI-generated data for model training and evaluation. As datasets grow, they often add semantically redundant samples rather than expanding semantic coverage, which deteriorates the quality of the whole dataset. Existing diversity metrics primarily summarize lexical variation, entropy, or pairwise similarity, but provide limited characterization of the shape of the underlying data distribution. In this paper, we propose PLDiv, a diversity metric derived from persistence landscapes that quantifies the structural organization of a dataset from its geometric representation. PLDiv summarizes how data neighborhoods evolve as nearby samples progressively connect, producing an interpretable scalar measure of structural diversity without requiring task-specific supervision. Whether measuring intrinsic diversity in text generations or image embeddings, or detecting semantic collapse in instruction datasets and reasoning trajectories, PLDiv performs competitively against evaluated baselines and aligns strongly with diversity labels during semantic collapse. Consequently, it effectively supports applications in synthetic data monitoring, dataset construction, data augmentation, and generative model evaluation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
