arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.05326cs.CL
收藏

基于心理学的统一动态框架用于课程学习

A Psychology-based Unified Dynamic Framework for Curriculum Learning

University of Notre Dame, Department of Computer Science and Engineering(诺丁汉大学计算机科学与工程系);Nankai University, College of Computer Science(南开大学计算机学院);University of Notre Dame, Department of IT, Analytics, and Operations(诺丁汉大学信息技术、分析与运营系);VA Bedford Healthcare System, Center for Health Optimization and Implementation Research(美国退伍军人事务部贝福德医疗系统健康优化与实施研究中心);University of Massachusetts Amherst, Manning College of Information and Computer Sciences(马萨诸塞大学阿姆赫斯特分校曼宁信息与计算机科学学院);University of Massachusetts Lowell, Miner School of Computer and Information Sciences(马萨诸塞大学洛威分校米纳尔计算机与信息科学学院)

Guangyu Meng, Qingkai Zeng, John P. Lalor, Hong Yu

AI总结:

本文提出基于心理学的统一动态框架用于课程学习,通过项目反应理论量化训练数据难度并优化数据选择,提升模型训练效率和准确性。

摘要:

直接从不同难度级别的示例中学习对人类和机器学习模型来说都具有挑战性。更有效的方法是将学习者逐步暴露于从简单到困难的示例中。课程学习(CL)已被提出用于实现这一策略在机器学习模型训练中。然而,在CL框架设计中仍存在两个关键挑战:定义训练数据的难度以及确定每个训练步骤应输入的数据量。受心理测量学的启发,本文提出了一种基于心理学的统一动态框架用于课程学习(PUDF)。我们通过将项目反应理论(IRT)应用于人工人群(AC)的响应来量化训练数据的难度。这种基于理论的IRT-AC方法导致了全局(即模型无关)且可解释的难度值。利用IRT,我们提出了一种训练策略,即通过模型能力估计的动态数据选择(DDS-MAE),以在模型训练过程中安排适当的数据量。由于我们的难度标注和模型能力估计是基于同一理论,即IRT,因此它们的值在相同范围内是可比的,这可能导致训练数据选择的一致性和更快的收敛速度,与其它CL方法相比。实验结果表明,使用PUDF对预训练的大型语言模型进行微调,在一系列基准数据集上实现了更高的准确性和更快的收敛速度,优于标准微调和最先进的CL方法。消融研究和下游分析进一步验证了PUDF对CL的影响。

英文摘要:

Directly learning from examples of varying difficulty levels is often challenging for both humans and machine learning models. A more effective strategy involves exposing learners to examples in a progressive order from easy to difficult. Curriculum Learning (CL) has been proposed to implement this strategy in machine learning model training. However, two key challenges persist in CL framework design: defining the difficulty of training data and determining the appropriate amount of data to input at each training step. Drawing inspiration from psychometrics, this paper presents a Psychology-based Unified Dynamic Framework for Curriculum Learning (PUDF). We quantify the difficulty of training data by applying Item Response Theory (IRT) to responses from Artificial Crowds (AC). This theory-driven IRT-AC approach leads to global (i.e., model-independent) and interpretable difficulty values. Leveraging IRT, we propose a training strategy, Dynamic Data Selection via Model Ability Estimation (DDS-MAE), to schedule the appropriate amount of data during model training. Since our difficulty labeling and model ability estimation are based on a consistent theory, namely IRT, their values are comparable within the same scope, potentially leading to aligned training data selection and faster convergence compared to the other CL methods. Experimental results demonstrate that fine-tuning pre-trained large language models with PUDF leads to higher accuracy and faster convergence on a suite of benchmark datasets compared to standard fine-tuning and state-of-the-art CL methods. Ablation studies and downstream analyses further validate the impact of PUDF for CL.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑