A Psychology-based Unified Dynamic Framework for Curriculum Learning
University of Notre Dame, Department of Computer Science and Engineering(诺丁汉大学计算机科学与工程系);Nankai University, College of Computer Science(南开大学计算机学院);University of Notre Dame, Department of IT, Analytics, and Operations(诺丁汉大学信息技术、分析与运营系);VA Bedford Healthcare System, Center for Health Optimization and Implementation Research(美国退伍军人事务部贝福德医疗系统健康优化与实施研究中心);University of Massachusetts Amherst, Manning College of Information and Computer Sciences(马萨诸塞大学阿姆赫斯特分校曼宁信息与计算机科学学院);University of Massachusetts Lowell, Miner School of Computer and Information Sciences(马萨诸塞大学洛威分校米纳尔计算机与信息科学学院)
Guangyu Meng, Qingkai Zeng, John P. Lalor, Hong Yu
备注:
Accepted for publication in Computational Linguistics. This is a pre-MIT Press publication version. Code available at https://github.com/nd-ball/cl-irt
Directly learning from examples of varying difficulty levels is often challenging for both humans and machine learning models. A more effective strategy involves exposing learners to examples in a progressive order from easy to difficult. Curriculum Learning (CL) has been proposed to implement this strategy in machine learning model training. However, two key challenges persist in CL framework design: defining the difficulty of training data and determining the appropriate amount of data to input at each training step. Drawing inspiration from psychometrics, this paper presents a Psychology-based Unified Dynamic Framework for Curriculum Learning (PUDF). We quantify the difficulty of training data by applying Item Response Theory (IRT) to responses from Artificial Crowds (AC). This theory-driven IRT-AC approach leads to global (i.e., model-independent) and interpretable difficulty values. Leveraging IRT, we propose a training strategy, Dynamic Data Selection via Model Ability Estimation (DDS-MAE), to schedule the appropriate amount of data during model training. Since our difficulty labeling and model ability estimation are based on a consistent theory, namely IRT, their values are comparable within the same scope, potentially leading to aligned training data selection and faster convergence compared to the other CL methods. Experimental results demonstrate that fine-tuning pre-trained large language models with PUDF leads to higher accuracy and faster convergence on a suite of benchmark datasets compared to standard fine-tuning and state-of-the-art CL methods. Ablation studies and downstream analyses further validate the impact of PUDF for CL.
RuCL: Stratified Rubric-Based Curriculum Learning for Multimodal Large Language Model Reasoning
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院);University of Chinese Academy of Sciences(中国科学院大学);Alibaba Group(阿里巴巴集团);School of Biomedical Engineering, UNSW Sydney(新南威尔士大学生物医学工程学院)
Actor-Curator: Co-adaptive Curriculum Learning via Policy-Improvement Bandits for RL Post-Training
University of Illinois Chicago(伊利诺伊大学香槟分校);Caltech(加州理工学院);RPI(罗切斯特理工学院);MBZUAI(澳门大学人工智能研究院);University of Chicago(芝加哥大学);NEC Laboratories America(NEC美国实验室)
Dynamic Data Selection for Curriculum Learning via Ability Estimation
Department of IT, Analytics, and Operations, University of Notre Dame(圣母大学IT、分析与运营系);Department of Computer Science, University of Massachusetts Lowell(马萨诸塞大学洛厄尔分校计算机科学系);College of Information and Computer Sciences, University of Massachusetts Amherst(马萨诸塞大学阿默斯特分校信息与计算机科学学院)