Curriculum learning is a training strategy that sorts the training examples by some measure of their difficulty and gradually exposes them to the learner to improve the network performance. Motivated by our insights from implicit curriculum ordering, we first introduce a simple curriculum learning strategy that uses statistical measures such as standard deviation and entropy values to score the difficulty of data points for real image classification tasks. We empirically show its improvements in performance with convolutional and fully-connected neural networks on multiple real image datasets. We also propose and study the performance of a dynamic curriculum learning algorithm. Our dynamic curriculum algorithm tries to reduce the distance between the network weight and an optimal weight at any training step by greedily sampling examples with gradients that are directed towards the optimal weight. Further, we use our algorithms to discuss why curriculum learning is helpful.
Does the Definition of Difficulty Matter? Scoring Functions and their Role for Curriculum Learning
Technical University of Munich（慕尼黑工业大学）；MCML – Munich Center for Machine Learning（慕尼黑机器学习中心）；Imperial College（帝国理工学院）；MDSI – Munich Data Science Institute（慕尼黑数据科学研究所）
Confusion-Aware Transfer Teacher Curriculum Learning Framework: Disentangling Scoring and Pacing Effects
Faculty of Computing, Sri Lanka Institute of Information Technology, Sri Lanka（斯里兰卡信息科技学院计算机学院，斯里兰卡）；Faculty of Engineering, University of Sri Jayewardenepura, Sri Lanka（斯里兰卡贾亚韦达内普拉大学工程学院，斯里兰卡）；Faculty of Engineering, Sri Lanka Institute of Information Technology, Sri Lanka（斯里兰卡信息科技学院工程学院，斯里兰卡）；University of Sheffield, United Kingdom（谢菲尔德大学，英国）；Utrecht University, The Netherlands（乌得勒支大学，荷兰）
Dual-Criterion Curriculum Learning: Application to Temporal Data
Université Paris Saclay, Université Paris Cité, ENS Paris Saclay, CNRS, SSA, INSERM, Centre Borelli（巴黎萨克雷大学、巴黎cité大学、巴黎萨克雷高等师范学院、国家科学研究中心、SSA、国家医学研究院、Borelli中心）；Centre d’Analyse et de Mathématique Sociales, EHESS, CNRS（分析与数学社会研究中心、高等社会科学学院、国家科学研究中心）；EDF R&D（电力公司研发）
