arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2205.09898cs.LGcs.CL
收藏

让模型决定多任务学习的课程

Let the Model Decide its Curriculum for Multitask Learning

Arizona State University(亚利桑那州立大学)

Neeraj Varshney, Swaroop Mishra, Chitta Baral

摘要:

现有多任务学习方法中的课程学习策略基于人类感知或通过穷尽地搜索最优布置来将数据集布置在难度层级中。然而,人类对难度的感知可能并不总是与机器解释很好地相关,导致性能差,并且穷举搜索在计算上是昂贵的。为了解决这些问题,我们提出了两类技术,根据通过基于模型的方法计算的难度分数将训练实例安排到学习课程中。这两个类,即数据集级和实例级,在排列粒度上有所不同。通过对12个数据集的全面实验,我们表明实例级和数据集级技术可以产生强有力的表示,因为它们比各自的基线平均性能提高了4.17%和3.15%。此外,我们发现,这种改善大部分来自正确回答困难的情况下,这意味着我们的技术在困难的任务更大的功效。

英文摘要:

Curriculum learning strategies in prior multi-task learning approaches arrange datasets in a difficulty hierarchy either based on human perception or by exhaustively searching the optimal arrangement. However, human perception of difficulty may not always correlate well with machine interpretation leading to poor performance and exhaustive search is computationally expensive. Addressing these concerns, we propose two classes of techniques to arrange training instances into a learning curriculum based on difficulty scores computed via model-based approaches. The two classes i.e Dataset-level and Instance-level differ in granularity of arrangement. Through comprehensive experiments with 12 datasets, we show that instance-level and dataset-level techniques result in strong representations as they lead to an average performance improvement of 4.17% and 3.15% over their respective baselines. Furthermore, we find that most of this improvement comes from correctly answering the difficult instances, implying a greater efficacy of our techniques on difficult tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑