CLIP驱动的鲁棒且可泛化的数据选择框架
A CLIP-Powered Framework for Robust and Generalizable Data Selection
AI总结：
提出CLIP驱动的多模态数据选择框架，通过数据集适应、样本评分和选择优化三个模块，利用预训练知识评估样本影响，在多个基准上优于现有方法，并能去除噪声样本以提升性能。
摘要：
近年来，大规模数据集对深度学习模型的进步起到了关键作用，但在如此大规模的数据集上进行训练不可避免地会产生大量的存储和计算开销。同时，现实世界的数据集往往包含冗余和噪声数据，对训练效率和模型性能产生负面影响。数据选择在识别整个数据集中最具代表性的样本方面显示出潜力，其目标是在降低训练成本的同时最小化性能差距。现有工作通常依赖单模态信息为单个样本分配重要性分数，这可能导致评估不准确，尤其是在处理噪声或损坏样本时。为解决这一局限，我们提出了一种新颖的CLIP驱动的数据选择框架，利用多模态信息实现更鲁棒且可泛化的样本选择。具体而言，我们的框架由三个关键模块组成——数据集适应、样本评分和选择优化——这些模块共同利用广泛的预训练多模态知识来全面评估样本影响，并通过多目标优化优化选择结果。大量实验表明，我们的方法在各种基准数据集上始终优于现有的最先进基线。值得注意的是，我们的方法有效去除了数据集中的噪声或损坏样本，使其能够用更少的数据实现更高的性能。这表明它不仅是一种加速训练的方式，还能提高整体数据质量。
英文摘要：
Large-scale datasets have been pivotal to the advancements of deep learning models in recent years, but training on such large datasets invariably incurs substantial storage and computational overhead. Meanwhile, real-world datasets often contain redundant and noisy data, imposing a negative impact on training efficiency and model performance. Data selection has shown promise in identifying the most representative samples from the entire dataset, which aims to minimize the performance gap with reduced training costs. Existing works typically rely on single-modality information to assign importance scores for individual samples, which may lead to inaccurate assessments, especially when dealing with noisy or corrupted samples. To address this limitation, we propose a novel CLIP-powered data selection framework that leverages multimodal information for more robust and generalizable sample selection. Specifically, our framework consists of three key modules-dataset adaptation, sample scoring, and selection optimization-that together harness extensive pre-trained multimodal knowledge to comprehensively assess sample influence and optimize the selection results through multi-objective optimization. Extensive experiments demonstrate that our approach consistently outperforms existing state-of-the-art baselines on various benchmark datasets. Notably, our method effectively removes noisy or damaged samples from the dataset, enabling it to achieve even higher performance with less data. This indicates that it is not only a way to accelerate training but can also improve overall data quality.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
