CovMatch：基于交叉协方差引导且配备可训练文本编码器的多模态数据集蒸馏
CovMatch: Cross-Covariance Guided Multimodal Dataset Distillation with Trainable Text Encoder
AI总结：
针对多模态数据集蒸馏中冻结文本编码器限制语义对齐的问题，提出CovMatch框架，通过对齐真实与合成特征交叉协方差并正则化模态内分布，支持双编码器联合优化，在Flickr30K和COCO上仅用500合成对实现最高6.8%的检索准确率绝对提升。
摘要：
多模态数据集蒸馏旨在合成一小组图文对，以实现大规模视觉-语言模型的高效训练。尽管数据集蒸馏在单模态任务中已展现出潜力，但将其扩展到多模态对比学习面临两大核心挑战：学习跨模态对齐，以及应对大型编码器的高计算成本。现有方法通过冻结文本编码器、仅更新图像编码器和文本投影层来解决可扩展性问题，但我们发现这会严重限制语义对齐能力，成为性能提升的瓶颈。我们提出CovMatch——一种可扩展的数据集蒸馏框架，它对齐真实与合成特征的交叉协方差，同时对各模态内的特征分布进行正则化。与现有方法不同，CovMatch支持两个编码器的联合优化，从而实现更强的跨模态对齐与更优的性能。在Flickr30K和COCO数据集上的评估表明，CovMatch性能优于当前最先进的多模态蒸馏方法，仅用500个合成图文对就实现了最高6.8%的检索准确率绝对提升。
英文摘要：
Multimodal dataset distillation aims to synthesize a small set of image-text pairs that enables efficient training of large-scale vision-language models. While dataset distillation has shown promise in unimodal tasks, extending it to multimodal contrastive learning presents key challenges: learning cross-modal alignment and managing the high computational cost of large encoders. Prior approaches address scalability by freezing the text encoder and update only the image encoder and text projection layer. However, we find this severely limits semantic alignment and becomes a bottleneck for performance scaling. We propose CovMatch, a scalable dataset distillation framework that aligns the cross-covariance of real and synthetic features while regularizing feature distributions within each modality. Unlike prior approaches, CovMatch enables joint optimization of both encoders, leading to stronger cross-modal alignment and improved performance. Evaluated on Flickr30K and COCO, CovMatch outperforms state-of-the-art multimodal distillation methods and achieves up to 6.8% absolute gains in retrieval accuracy using only 500 synthetic pairs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
