arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2311.18531cs.CVcs.AIcs.LG
收藏

基于Wasserstein度量的数据集蒸馏

Dataset Distillation via the Wasserstein Metric

University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);University of California, San Diego(加州大学圣地亚哥分校);National University of Singapore(新加坡国立大学);University of California, Los Angeles(加州大学洛杉矶分校);Sri Aurobindo International Centre of Education(斯里阿罗bindo国际教育中心)

Haoyang Liu, Yijiang Li, Tiancheng Xing, Peiran Wang, Vibhu Dalal, Luwei Li, Jingrui He, Haohan Wang

AI总结:

本文提出WMDD方法,利用Wasserstein重心和逐类BatchNorm统计量优化合成数据分布匹配,在保持高效性的同时于多个高分辨率数据集上取得最先进性能。

摘要:

数据集蒸馏(Dataset Distillation,DD)旨在生成一个紧凑的合成数据集,使模型能够达到与在完整大数据集上训练相媲美的性能,从而显著降低计算成本。借鉴最优传输理论,我们提出了WMDD(Wasserstein Metric-based Dataset Distillation,基于Wasserstein度量的数据集蒸馏),这是一种简洁而强大的方法,利用Wasserstein度量来增强分布匹配。我们计算预训练分类器特征的Wasserstein重心,以捕捉原始数据分布的本质特征。通过在特征空间中优化合成数据以与该重心对齐,并利用逐类BatchNorm统计量来保持类内变化,WMDD在保持分布匹配方法高效性的同时,在多个高分辨率数据集上取得了最先进的结果。我们的大量实验证明了WMDD的有效性和适应性,凸显了其在大规模机器学习应用中的潜力。

英文摘要:

Dataset Distillation (DD) aims to generate a compact synthetic dataset that enables models to achieve performance comparable to training on the full large dataset, significantly reducing computational costs. Drawing from optimal transport theory, we introduce WMDD (Wasserstein Metric-based Dataset Distillation), a straightforward yet powerful method that employs the Wasserstein metric to enhance distribution matching. We compute the Wasserstein barycenter of features from a pretrained classifier to capture essential characteristics of the original data distribution. By optimizing synthetic data to align with this barycenter in feature space and leveraging per-class BatchNorm statistics to preserve intra-class variations, WMDD maintains the efficiency of distribution matching approaches while achieving state-of-the-art results across various high-resolution datasets. Our extensive experiments demonstrate WMDD's effectiveness and adaptability, highlighting its potential for advancing machine learning applications at scale.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑