通过生成式增强提升AI测试与评估中的语义覆盖度
Improving Semantic Coverage in AI Test & Evaluation through Generative Augmentation
AI总结：
该研究提出三阶段生成式AI流水线，利用扩散模型生成合成数据填补AI测试集的语义缺口，提升测试评估结果的准确性，无需模型重训练。
摘要：
在AI模型的测试与评估（T&E）中，评估者通常仅能获取预留测试数据集，而无法获知训练数据或其语义构成。若该测试数据集中关键输入条件（如一天中的时段、天气或场景位置）的代表性不足，T&E结果会因这种语义覆盖缺口而产生误导。我们提出了一种三阶段生成式AI流水线，用于检测并填补此类语义缺口，且无需这些语义的真实标签。第一阶段：视觉语言模型（VLMs）对语义因素执行零样本二分类标注，嵌入空间聚类步骤识别出对下游模型行为影响最大的语义；第二阶段：基于扩散的图像到图像模型将现有已覆盖的图像转换为代表性不足的语义域，在保留场景内容的同时增强高层外观；第三阶段：对增强后的图像嵌入进行统计分析，在将合成图像纳入测试集前验证其是否提升了数据集覆盖度。关键在于，合成数据仅用于丰富测试集，T&E过程中不执行任何模型重训练。我们在图像分类和目标检测任务的昼夜语义缺口场景下验证了完整流水线，结果表明，增强后的测试数据集能更好地区分是否接触过缺失语义的模型，从而得出与理想的完全覆盖数据集结论更接近的T&E结果。代码可访问此https URL
英文摘要：
In test and evaluation (T&E) of AI models, evaluators typically have access only to a held-out test dataset and no knowledge of the training data or its semantic composition. If critical input conditions---such as time-of-day, weather, or scene location---are underrepresented in this test dataset, T&E results can be misleadingly due to this semantic coverage gap. We propose a three-stage generative AI pipeline to detect and fill such semantic gaps without requiring ground-truth labels of these semantics. First, vision-language models (VLMs) perform zero-shot binary labeling of semantic factors, and an embedding-space clustering step identifies which semantics most affect downstream model behavior. Second, diffusion-based image-to-image models transform existing covered images into the underrepresented semantic domain, preserving scene content while augmenting high-level appearance. Third, statistical analysis of the augmented image embeddings verifies that the synthetic imagery improves dataset coverage before it is incorporated. Crucially, the synthetic data is used only to enrich the test set and no model retraining is performed for T&E. We validate the full pipeline on both image classification and object detection under day/night semantic gaps, showing that the augmented test datasets better distinguish models with and without exposure to the missing semantic, thereby yielding T&E conclusions that more closely match those of an ideal, fully covered dataset. Code available at https://github.com/Hashe037/DEDUCE
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
