野外多模态数据集蒸馏
Multi-Modal Dataset Distillation in the Wild
AI总结：
本文提出首个野外多模态数据集蒸馏框架MDW，通过可学习细粒度对应关系和双轨协同学习，将含噪多模态数据蒸馏为紧凑干净的数据集，在多种压缩比下超越先前方法超过15%。
摘要：
近年来，多模态模型在现实世界应用中展现出卓越的通用性。然而，其快速发展面临两大关键数据挑战。首先，训练过程需要大规模数据集，导致巨大的存储和计算成本。其次，这些数据通常通过网络爬取获得，不可避免地带有噪声，即部分不匹配的配对数据，严重损害模型性能。针对这些问题，我们提出了野外多模态数据集蒸馏框架MDW，这是首个将含噪多模态数据集蒸馏为紧凑且干净的数据集、以实现高效且有效的模型训练的框架。具体而言，MDW在蒸馏过程中引入可学习的细粒度对应关系，并自适应地优化蒸馏数据以突出具有对应判别能力的区域，从而提升蒸馏数据的信息密度和有效性。此外，为了从真实数据中捕获鲁棒的跨模态对应先验知识，MDW提出双轨协同学习以规避有风险的数据噪声，在具备可证明的噪声容忍性的同时缓解信息损失。大量实验验证了MDW在理论和实证上的有效性以及显著的可扩展性，在各种压缩比下均超越先前方法超过15%，凸显了其在具有不同效能和资源需求的应用中极具吸引力的实用性。
英文摘要：
Recent multi-modal models have shown remarkable versatility in real-world applications. However, their rapid development encounters two critical data challenges. First, the training process requires large-scale datasets, leading to substantial storage and computational costs. Second, these data are typically web-crawled with inevitable noise, i.e., partially mismatched pairs, severely degrading model performance. To these ends, we propose Multi-modal dataset Distillation in the Wild, i.e., MDW, the first framework to distill noisy multi-modal datasets into compact clean ones for effective and efficient model training. Specifically, MDW introduces learnable fine-grained correspondences during distillation and adaptively optimizes distilled data to emphasize correspondence-discriminative regions, thereby enhancing distilled data's information density and efficacy. Moreover, to capture robust cross-modal correspondence prior knowledge from real data, MDW proposes dual-track collaborative learning to avoid the risky data noise, alleviating information loss with certifiable noise tolerance. Extensive experiments validate MDW's theoretical and empirical efficacy with remarkable scalability, surpassing prior methods by over 15% across various compression ratios, highlighting its appealing practicality for applications with diverse efficacy and resource needs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
