arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2502.21147cs.LGcs.CV
收藏

相同精度,两倍速度:持续训练优于从头再训练

Same accuracy, twice as fast: continuous training surpasses retraining from scratch

KU Leuven(鲁汶大学)

Eli Verwimp, Guy Hacohen, Tinne Tuytelaars

AI总结:

针对新旧数据均可用的场景,提出利用先前模型和旧数据的持续训练方法,通过优化初始化、正则化、数据选择和超参数,在保持精度的同时实现高达2.7倍的计算加速。

摘要:

持续学习旨在使模型能够适应新数据集,同时不损失在先前学习数据上的性能,通常假设先前数据不再可用。然而,在许多实际场景中,旧数据和新数据都是可获取的。在这种情况下,要在两个数据集上都取得良好性能,通常的做法是放弃在先前数据上训练的模型,并在两个数据集上从头重新训练一个新模型。这种从头训练在计算上代价高昂。相比之下,利用先前训练的模型和旧数据的方法值得研究,因为它们可以显著降低计算成本。我们的评估框架量化了此类方法在保持或超越从头训练性能的同时所节省的计算量。我们确定了关键的优化方面——初始化、正则化、数据选择和超参数——每个方面都可以有助于降低计算成本。针对每个方面,我们提出了有效的初步方法,这些方法已经产生了可观的计算节省。通过结合这些方法,我们在各种计算机视觉任务中实现了高达2.7倍的计算时间缩减,凸显了该领域进一步发展的潜力。

英文摘要:

Continual learning aims to enable models to adapt to new datasets without losing performance on previously learned data, often assuming that prior data is no longer available. However, in many practical scenarios, both old and new data are accessible. In such cases, good performance on both datasets is typically achieved by abandoning the model trained on the previous data and re-training a new model from scratch on both datasets. This training from scratch is computationally expensive. In contrast, methods that leverage the previously trained model and old data are worthy of investigation, as they could significantly reduce computational costs. Our evaluation framework quantifies the computational savings of such methods while maintaining or exceeding the performance of training from scratch. We identify key optimization aspects -- initialization, regularization, data selection, and hyper-parameters -- that can each contribute to reducing computational costs. For each aspect, we propose effective first-step methods that already yield substantial computational savings. By combining these methods, we achieve up to 2.7x reductions in computation time across various computer vision tasks, highlighting the potential for further advancements in this area.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑