arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2202.05531cs.LGcs.AI
收藏

循环式课程学习

Cyclical Curriculum Learning

Yildiz Technical University(耶尔德兹理工大学)

H. Toprak Kesgin, M. Fatih Amasyali

摘要:

人工神经网络(ANN)是受人类学习的启发而产生的。然而,与人类教育不同,经典ANN不使用课程。课程学习(CL)是指人工神经网络的训练过程,其中的例子是在一个有意义的顺序。当使用CL时,训练从数据集的一个子集开始,并在整个训练过程中添加新样本,或者训练从整个数据集开始,并减少使用的样本数量。随着训练数据集大小的这些变化,使用课程,反课程或随机课程方法可以获得比香草方法更好的结果。然而,一个普遍有效的CL方法为各种架构和数据sets.In本文中,我们提出了循环课程学习(CCL),在训练过程中使用的数据大小循环变化,而不是简单地增加或减少。而不是只使用香草的方法或只使用课程的方法,使用这两种方法循环像在CCL提供了更成功的结果。我们在18个不同的数据集和15个图像和文本分类任务的架构上测试了该方法,并获得了比无CL和现有CL方法更成功的结果。我们还从理论上证明了循环应用CL和vanilla而不是仅使用CL或仅使用vanilla方法的错误较少。循环课程的代码可在https://github.com/CyclicalCurriculum/Cyclical-Curriculum上找到。

英文摘要:

Artificial neural networks (ANN) are inspired by human learning. However, unlike human education, classical ANN does not use a curriculum. Curriculum Learning (CL) refers to the process of ANN training in which examples are used in a meaningful order. When using CL, training begins with a subset of the dataset and new samples are added throughout the training, or training begins with the entire dataset and the number of samples used is reduced. With these changes in training dataset size, better results can be obtained with curriculum, anti-curriculum, or random-curriculum methods than the vanilla method. However, a generally efficient CL method for various architectures and data sets is not found. In this paper, we propose cyclical curriculum learning (CCL), in which the data size used during training changes cyclically rather than simply increasing or decreasing. Instead of using only the vanilla method or only the curriculum method, using both methods cyclically like in CCL provides more successful results. We tested the method on 18 different data sets and 15 architectures in image and text classification tasks and obtained more successful results than no-CL and existing CL methods. We also have shown theoretically that it is less erroneous to apply CL and vanilla cyclically instead of using only CL or only vanilla method. The code of Cyclical Curriculum is available at https://github.com/CyclicalCurriculum/Cyclical-Curriculum.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑