在线持续学习中应对合成数据污染问题
Dealing with Synthetic Data Contamination in Online Continual Learning
AI总结：
本文研究合成数据污染对在线持续学习的影响，提出ESRM方法，通过熵选择和真实-合成相似性最大化来缓解合成图像导致的性能下降，实验证明其在严重污染下效果显著。
摘要：
图像生成技术在生成高保真逼真图像方面取得了显著成果，尤其是随着基于扩散的模型的进步。然而，AI生成图像的普及可能对机器学习社区产生尚未被明确识别的副作用。与此同时，深度学习在计算机视觉领域的成功是由互联网上收集的海量数据集所推动的。大量合成数据被添加到互联网上，将成为未来研究人员收集不含AI生成内容的“干净”数据集的障碍。先前的研究表明，使用被合成图像污染的数据集进行训练可能导致性能下降。在本文中，我们研究了受污染数据集对在线持续学习（Online Continual Learning，CL）研究的潜在影响。我们通过实验表明，受污染的数据集可能会阻碍现有在线CL方法的训练。此外，我们提出了熵选择与真实-合成相似性最大化（Entropy Selection with Real-synthetic similarity Maximization，ESRM）方法，以缓解在训练在线CL模型时由合成图像引起的性能恶化。实验表明，我们的方法可以显著缓解性能恶化，尤其是在污染严重的情况下。为了可复现性，我们工作的源代码可在 https://github.com/maorong-wang/ESRM 获取。
英文摘要：
Image generation has shown remarkable results in generating high-fidelity realistic images, in particular with the advancement of diffusion-based models. However, the prevalence of AI-generated images may have side effects for the machine learning community that are not clearly identified. Meanwhile, the success of deep learning in computer vision is driven by the massive dataset collected on the Internet. The extensive quantity of synthetic data being added to the Internet would become an obstacle for future researchers to collect "clean" datasets without AI-generated content. Prior research has shown that using datasets contaminated by synthetic images may result in performance degradation when used for training. In this paper, we investigate the potential impact of contaminated datasets on Online Continual Learning (CL) research. We experimentally show that contaminated datasets might hinder the training of existing online CL methods. Also, we propose Entropy Selection with Real-synthetic similarity Maximization (ESRM), a method to alleviate the performance deterioration caused by synthetic images when training online CL models. Experiments show that our method can significantly alleviate performance deterioration, especially when the contamination is severe. For reproducibility, the source code of our work is available at https://github.com/maorong-wang/ESRM.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
