节食式持续学习：在受限计算下从稀疏标注流中学习
Continual Learning on a Diet: Learning from Sparsely Labeled Streams Under Constrained Computation
AI总结：
本文提出DietCL，针对计算预算受限且标注稀疏的持续学习场景，通过联合利用标注与未标注数据并精心分配计算资源，在多个大规模数据集上显著超越现有监督和半监督持续学习方法。
摘要：
我们提出并研究了一种现实的持续学习（CL）设置，其中学习算法在训练的每个时间步被赋予受限的计算预算。我们将该设置应用于具有稀疏标注率的大规模半监督持续学习场景。先前熟练的CL方法在这一具有挑战性的设置中表现非常差。对稀疏标注数据的过拟合和计算预算不足是导致如此差性能的两个主要原因。我们的新设置鼓励学习方法在训练期间有效且高效地利用未标注数据。为此，我们提出了一个简单但高效的基线方法DietCL，它联合利用未标注和标注数据。DietCL精心为两类数据分配计算预算。我们在多个数据集上（例如CLOC、ImageNet10K和CGLM）在受限预算设置下大规模验证了我们的基线。DietCL大幅优于所有现有的监督CL算法以及更新的持续半监督方法。我们广泛的分析和消融实验表明，DietCL在完整的标注稀疏度、计算预算以及各种其他消融条件下都是稳定的。
英文摘要：
We propose and study a realistic Continual Learning (CL) setting where learning algorithms are granted a restricted computational budget per time step while training. We apply this setting to large-scale semi-supervised Continual Learning scenarios with sparse label rates. Previous proficient CL methods perform very poorly in this challenging setting. Overfitting to the sparse labeled data and insufficient computational budget are the two main culprits for such a poor performance. Our new setting encourages learning methods to effectively and efficiently utilize the unlabeled data during training. To that end, we propose a simple but highly effective baseline, DietCL, which utilizes both unlabeled and labeled data jointly. DietCL meticulously allocates computational budget for both types of data. We validate our baseline, at scale, on several datasets, e.g., CLOC, ImageNet10K, and CGLM, under constraint budget setups. DietCL outperforms, by a large margin, all existing supervised CL algorithms as well as more recent continual semi-supervised methods. Our extensive analysis and ablations demonstrate that DietCL is stable under a full spectrum of label sparsity, computational budget, and various other ablations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
