重新思考长尾数据集蒸馏：一种统一层面框架，具有无偏恢复和重标记
Rethinking Long-tailed Dataset Distillation: A Uni-Level Framework with Unbiased Recovery and Relabeling
AI总结：
本文提出一种统一框架，通过无偏恢复和重标记方法提升长尾数据集蒸馏效果，实验显示在多个基准上显著提高准确率。
摘要：
数据集蒸馏创建了一个小的蒸馏集，通过捕捉完整数据集的关键信息来实现高效训练。尽管现有的数据集蒸馏方法在平衡数据集上表现良好，但在长尾分布下却面临困难，不平衡的类别频率导致模型表示偏差，并破坏统计估计，如批量归一化（BN）统计。在本文中，我们通过重新审视轨迹方法的局限性，重新思考长尾数据集蒸馏，采用统计对齐视角，共同缓解模型偏差并恢复公平监督。为此，我们引入三个专门组件，使蒸馏图像的无偏恢复和软重标记成为可能：（1）增强专家模型（一个用于恢复的观察模型和一个用于重标记的教师模型）以实现可靠的统计估计和软标签生成；（2）通过全前向传递并动态调整动量来重新校准BN统计，以减少表示偏斜；（3）通过多轮机制逐步选择高置信度和多样化的增强，初始化合成图像，促进覆盖和多样性。在四个长尾基准上的大量实验显示，在不同类别不平衡程度下，与现有最先进方法相比，有持续的改进。值得注意的是，我们的方法在IPC=10和IF=10的情况下，CIFAR-100-LT的top-1准确率提高了15.6%，Tiny-ImageNet-LT提高了11.8%。代码可在https://github.com/2018cx/RLDD上获得。
英文摘要：
Dataset distillation creates a small distilled set that enables efficient training by capturing key information from the full dataset. While existing dataset distillation methods perform well on balanced datasets, they struggle under long-tailed distributions, where imbalanced class frequencies induce biased model representations and corrupt statistical estimates such as Batch Normalization (BN) statistics. In this paper, we rethink long-tailed dataset distillation by revisiting the limitations of trajectory-based methods, and instead adopt the statistical alignment perspective to jointly mitigate model bias and restore fair supervision. To this end, we introduce three dedicated components that enable unbiased recovery of distilled images and soft relabeling: (1) enhancing expert models (an observer model for recovery and a teacher model for relabeling) to enable reliable statistics estimation and soft-label generation; (2) recalibrating BN statistics via a full forward pass with dynamically adjusted momentum to reduce representation skew; (3) initializing synthetic images by incrementally selecting high-confidence and diverse augmentations via a multi-round mechanism that promotes coverage and diversity. Extensive experiments on four long-tailed benchmarks show consistent improvements over state-of-the-art methods across varying degrees of class imbalance. Notably, our approach improves top-1 accuracy by 15.6% on CIFAR-100-LT and 11.8% on Tiny-ImageNet-LT under IPC=10 and IF=10. Codes are available at https://github.com/2018cx/RLDD.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
