arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2208.09129cs.CL
收藏

从粗到精:用于自然语言理解的分层多任务学习

Coarse-to-Fine: Hierarchical Multi-task Learning for Natural Language Understanding

Huawei Poisson Lab(华为泊松实验室);University of Montreal(蒙特利尔大学);Renmin University of China(中国人民大学);Fudan University(复旦大学)

Zhaoye Fei, Yu Tian, Yongkang Wu, Xinyu Zhang, Yutao Zhu, Zheng Liu, Jiawen Wu, Dejiang Kong, Ruofei Lai, Zhao Cao, Zhicheng Dou, Xipeng Qiu

摘要:

广义文本表示是许多自然语言理解任务的基础。为了充分利用不同的语料库,模型不可避免地需要了解它们之间的相关性。然而,许多方法忽略了相关性,直接对所有任务采用单通道模型(粗糙范式),缺乏足够的合理性和解释性。此外,现有的一些工作通过缝合技能块(一个很好的范例)来学习下游任务,由于其冗余和噪声,可能会导致不合理的结果。在这项工作中,我们首先从三个不同的角度来分析任务相关性,即,数据属性、手动设计和基于模型的相关性,基于这些相似的任务被分组在一起。然后,我们提出了一个分层的框架与粗到细的范例,与底层共享的所有任务,中间层划分到不同的组,和顶层分配给每个任务。这使得我们的模型能够从所有任务中学习基本的语言属性,提高相关任务的性能,并减少不相关任务的负面影响。我们在13个基准数据集上进行了5个自然语言理解任务的实验,证明了我们方法的优越性。

英文摘要:

Generalized text representations are the foundation of many natural language understanding tasks. To fully utilize the different corpus, it is inevitable that models need to understand the relevance among them. However, many methods ignore the relevance and adopt a single-channel model (a coarse paradigm) directly for all tasks, which lacks enough rationality and interpretation. In addition, some existing works learn downstream tasks by stitches skill block(a fine paradigm), which might cause irrationalresults due to its redundancy and noise. Inthis work, we first analyze the task correlation through three different perspectives, i.e., data property, manual design, and model-based relevance, based on which the similar tasks are grouped together. Then, we propose a hierarchical framework with a coarse-to-fine paradigm, with the bottom level shared to all the tasks, the mid-level divided to different groups, and the top-level assigned to each of the tasks. This allows our model to learn basic language properties from all tasks, boost performance on relevant tasks, and reduce the negative impact from irrelevant tasks. Our experiments on 13 benchmark datasets across five natural language understanding tasks demonstrate the superiority of our method.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑