Inspired by human learning, researchers have proposed ordering examples during training based on their difficulty. Both curriculum learning, exposing a network to easier examples early in training, and anti-curriculum learning, showing the most difficult examples first, have been suggested as improvements to the standard i.i.d. training. In this work, we set out to investigate the relative benefits of ordered learning. We first investigate the \emph{implicit curricula} resulting from architectural and optimization bias and find that samples are learned in a highly consistent order. Next, to quantify the benefit of \emph{explicit curricula}, we conduct extensive experiments over thousands of orderings spanning three kinds of learning: curriculum, anti-curriculum, and random-curriculum -- in which the size of the training dataset is dynamically increased over time, but the examples are randomly ordered. We find that for standard benchmark datasets, curricula have only marginal benefits, and that randomly ordered samples perform as well or better than curricula and anti-curricula, suggesting that any benefit is entirely due to the dynamic training set size. Inspired by common use cases of curriculum learning in practice, we investigate the role of limited training time budget and noisy data in the success of curriculum learning. Our experiments demonstrate that curriculum, but not anti-curriculum can indeed improve the performance either with limited training time budget or in existence of noisy data.
Learning Multi-Objective Curricula for Robotic Policy Learning
McGill University(麦吉尔大学);Mila - Québec AI Institute(米拉-魁北克人工智能研究所);IBM Research(IBM研究院);Université de Montréal(蒙特利尔大学);Polytechnique Montréal(蒙特利尔理工学院);Beijing Academy of AI(北京人工智能研究院)
An Analytical Theory of Curriculum Learning in Teacher-Student Networks
Bocconi University(博科尼大学);Gatsby Computational Neuroscience Unit & Sainsbury Wellcome Centre, University College London(盖茨比计算神经科学单位与塞恩斯伯里惠康中心,伦敦大学学院);FAIR, Meta AI(FAIR,Meta AI)
On the Benefit of Optimal Transport for Curriculum Reinforcement Learning
Technical University of Darmstadt(达姆施塔特工业大学);German Research Center for AI (DFKI)(德国人工智能研究中心);Centre for Cognitive Science at Technical University of Darmstadt(达姆施塔特工业大学认知科学中心);Center for Artificial Intelligence and Data Science at University of Würzburg(维尔茨堡大学人工智能与数据科学中心);Aalto University(阿尔托大学)