arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2106.04072cs.AIcs.LG
收藏

由粗到精的课程学习

Coarse-to-Fine Curriculum Learning

Carnegie Mellon University(卡内基梅隆大学);Microsoft Semantic Machines(微软语义机器公司)

Otilia Stretcu, Emmanouil Antonios Platanios, Tom M. Mitchell, Barnabás Póczos

摘要:

当面临学习具有挑战性的新任务时,人类通常遵循一系列步骤,这些步骤使他们能够逐步建立执行这些新任务所需的技能。然而,在机器学习中,模型通常是直接训练来解决目标任务的。受人类学习的启发,我们提出了一种新的课程学习方法,该方法将具有挑战性的任务分解为一系列更容易的中间目标,用于在处理目标任务之前预训练模型。我们专注于分类任务,并使用自动构造的标签层次结构设计的中间任务。我们在层次结构的每个级别上对模型进行训练,从粗略的标签到精细的标签,在这些级别之间传输所获得的知识。例如,模型将首先学习区分动物和物体,然后在学习分类更细粒度的类(如猫、狗、汽车和卡车)时使用这些获得的知识。大多数现有的监督学习的课程学习算法包括调度训练样本被呈现给模型的顺序。相比之下,我们的方法侧重于模型的输出空间。我们在几个已建立的数据集上评估了我们的方法,并显示出显着的性能增益,特别是在具有许多标签的分类问题上。我们还评估了一个新的合成数据集,这使我们能够研究我们方法的多个方面。

英文摘要:

When faced with learning challenging new tasks, humans often follow sequences of steps that allow them to incrementally build up the necessary skills for performing these new tasks. However, in machine learning, models are most often trained to solve the target tasks directly.Inspired by human learning, we propose a novel curriculum learning approach which decomposes challenging tasks into sequences of easier intermediate goals that are used to pre-train a model before tackling the target task. We focus on classification tasks, and design the intermediate tasks using an automatically constructed label hierarchy. We train the model at each level of the hierarchy, from coarse labels to fine labels, transferring acquired knowledge across these levels. For instance, the model will first learn to distinguish animals from objects, and then use this acquired knowledge when learning to classify among more fine-grained classes such as cat, dog, car, and truck. Most existing curriculum learning algorithms for supervised learning consist of scheduling the order in which the training examples are presented to the model. In contrast, our approach focuses on the output space of the model. We evaluate our method on several established datasets and show significant performance gains especially on classification problems with many labels. We also evaluate on a new synthetic dataset which allows us to study multiple aspects of our method.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑