Dataset Condensation (DC) aims to reduce deep neural networks training efforts by synthesizing a small dataset such that it will be as effective as the original large dataset. Conventionally, DC relies on a costly bi-level optimization which prohibits its practicality. Recent research formulates DC as a distribution matching problem which circumvents the costly bi-level optimization. However, this efficiency sacrifices the DC performance. To investigate this performance degradation, we decomposed the dataset distribution into content and style. Our observations indicate two major shortcomings of: 1) style discrepancy between original and condensed data, and 2) limited intra-class diversity of condensed dataset. We present a simple yet effective method to match the style information between original and condensed data, employing statistical moments of feature maps as well-established style indicators. Moreover, we enhance the intra-class diversity by maximizing the Kullback-Leibler divergence within each synthetic class, i.e., content. We demonstrate the efficacy of our method through experiments on diverse datasets of varying size and resolution, achieving improvements of up to 4.1% on CIFAR10, 4.2% on CIFAR100, 4.3% on TinyImageNet, 2.0% on ImageNet-1K, 3.3% on ImageWoof, 2.5% on ImageNette, and 5.5% in continual learning accuracy.
M3D: Dataset Condensation by Minimizing Maximum Mean Discrepancy
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Shanghai University（上海大学）
DANCE: Dual-View Distribution Alignment for Dataset Condensation
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）；School of Computer Science, Fudan University（复旦大学计算机学院）
Elucidating the Design Space of Dataset Condensation
Mohamed bin Zayed University of AI（穆罕默德·本·扎耶德人工智能大学）；Tsinghua University（清华大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
Decomposed Global Optimization for Robust Point Matching with Low-Dimensional Branching
Department of Computer Science, Changzhi University, Changzhi, Shanxi, China, 046011（计算机科学系，长治大学）；School of Computer Science and Technology, Harbin Institute of Technology, Harbin 150001, China（计算机科学与技术学院，哈尔滨工业大学）
The Evolution of Dataset Distillation: Toward Scalable and Generalizable Solutions
Department of Computer Science and Engineering, University of Nevada, Reno（计算机科学与工程系，内华达大学雷诺分校）；Centre for Frontier AI Research (CFAR), and Institute of High Performance Computing (IHPC), A*STAR, Singapore（前沿人工智能研究中心（CFAR）和高性能计算研究所（IHPC），A*STAR，新加坡）
Towards Efficient Deep Hashing Retrieval: Condensing Your Data via Feature-Embedding Matching
College of Information Science and Technology, Donghua University（东华大学信息科学与技术学院）；ETH Zurich（苏黎世联邦理工学院）；College of Electrical Engineering, Zhejiang University（浙江大学电气工程学院）
