数据集蒸馏器是野生环境中优秀的标签去噪器
Dataset Distillers Are Good Label Denoisers In the Wild
AI总结：
针对传统含噪数据学习方法因初始噪声评估不准确陷入恶性循环的问题，提出基于数据集蒸馏的噪声去除方法，通过离线处理避免反馈循环、提升效率并保护隐私，实验验证其在随机噪声场景的去噪效果及高隐私环境的应用潜力。
摘要：
从含噪数据中学习对于将深度学习模型应用于实际场景至关重要。传统方法通常先评估噪声，再采用丢弃含噪样本、重新加权或重新标记等策略，但初始噪声评估不准确时会陷入恶性循环，导致性能不佳。为此，我们提出一种利用数据集蒸馏去除噪声的新方法，该方法避免现有技术常见的反馈循环，提升训练效率，且通过离线处理提供强隐私保护。我们在对称噪声、非对称噪声和真实自然噪声等多种噪声条件下，严格评估了三种代表性数据集蒸馏方法（DATM、DANCE和RCIG）。实验发现，数据集蒸馏在随机噪声场景中可有效去噪，但难以处理结构化非对称噪声（易被蒸馏样本吸收）；此外，干净但具有挑战性的样本（如不平衡数据集的尾部类样本）在蒸馏中可能出现有损压缩。尽管存在这些挑战，结果表明数据集蒸馏在鲁棒模型训练中具有重要潜力，尤其适用于噪声普遍存在的高隐私环境。源代码见https://github.com/Kciiiman/DD_LNL。
英文摘要：
Learning from noisy data has become essential for adapting deep learning models to real-world applications. Traditional methods often involve first evaluating the noise and then applying strategies such as discarding noisy samples, re-weighting, or re-labeling. However, these methods can fall into a vicious cycle when the initial noise evaluation is inaccurate, leading to suboptimal performance. To address this, we propose a novel approach that leverages dataset distillation for noise removal. This method avoids the feedback loop common in existing techniques and enhances training efficiency, while also providing strong privacy protection through offline processing. We rigorously evaluate three representative dataset distillation methods (DATM, DANCE, and RCIG) under various noise conditions, including symmetric noise, asymmetric noise, and real-world natural noise. Our empirical findings reveal that dataset distillation effectively serves as a denoising tool in random noise scenarios but may struggle with structured asymmetric noise patterns, which can be absorbed into the distilled samples. Additionally, clean but challenging samples, such as those from tail classes in imbalanced datasets, may undergo lossy compression during distillation. Despite these challenges, our results highlight that dataset distillation holds significant promise for robust model training, especially in high-privacy environments where noise is prevalent. The source code is available at https://github.com/Kciiiman/DD_LNL.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
