面向多模态数据集蒸馏的低秩相似性挖掘
Low-Rank Similarity Mining for Multimodal Dataset Distillation
AI总结：
本文提出低秩相似性挖掘（LoRS）方法，通过同时蒸馏图像-文本对和真实相似性矩阵并利用低秩分解，解决多模态数据集蒸馏中模态对应关系建模的难题，显著提升视觉-语言数据集蒸馏效果。
摘要：
尽管数据集蒸馏近年来发展迅速，但多模态数据（如图像-文本对）的蒸馏仍面临独特且尚未充分探索的挑战。与单模态数据不同，图像-文本对比学习（ITC）数据缺乏固有的类别划分，因此应更加注重模态间的对应关系。本文提出面向多模态数据集蒸馏的低秩相似性挖掘（LoRS）方法，该方法在蒸馏图像-文本对的同时蒸馏真实相似性矩阵，并利用低秩分解提升效率和可扩展性。所提方法显著改进了现有算法，为视觉-语言数据集蒸馏领域做出了重要贡献。我们倡导将LoRS作为图像-文本数据集蒸馏的基础合成数据设置。代码已开源：https://github.com/silicx/LoRS_Distill。
英文摘要：
Though dataset distillation has witnessed rapid development in recent years, the distillation of multimodal data, e.g., image-text pairs, poses unique and under-explored challenges. Unlike unimodal data, image-text contrastive learning (ITC) data lack inherent categorization and should instead place greater emphasis on modality correspondence. In this work, we propose Low-Rank Similarity Mining (LoRS) for multimodal dataset distillation, that concurrently distills a ground truth similarity matrix with image-text pairs, and leverages low-rank factorization for efficiency and scalability. The proposed approach brings significant improvement to the existing algorithms, marking a significant contribution to the field of visual-language dataset distillation. We advocate adopting LoRS as a foundational synthetic data setup for image-text dataset distillation. Our code is available at https://github.com/silicx/LoRS_Distill.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
