arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2110.07855cs.CL
收藏

面向AMR语法分析的分层课程学习

Hierarchical Curriculum Learning for AMR Parsing

Peking University(北京大学);Tencent Cloud Xiaowei(腾讯云小微)

Peiyi Wang, Liang Chen, Tianyu Liu, Damai Dai, Yunbo Cao, Baobao Chang, Zhifang Sui

摘要:

摘要意义表示(AMR)解析旨在将句子转换为具有层次结构的语义表示,最近通过预训练的序列到序列模型进行了授权。然而,他们的扁平化培训目标(即,平等对待所有输出令牌)和分层AMR结构,这限制了模型的泛化。为了弥补这一差距,我们提出了一个分层的课程学习(HCL)的框架,结构级(SC)和实例级课程(IC)。在训练过程中,SC逐渐从核心AMR语义元素切换到细节AMR语义元素,而IC从结构简单的AMR实例过渡到复杂的AMR实例。通过这两个预热过程,HCL降低了学习复杂结构的难度,从而使平面模型更好地适应AMR层次结构。在AMR2.0、AMR3.0、结构复杂和分布外的情况下进行了大量的实验,验证了HCL的有效性。

英文摘要:

Abstract Meaning Representation (AMR) parsing aims to translate sentences to semantic representation with a hierarchical structure, and is recently empowered by pretrained sequence-to-sequence models. However, there exists a gap between their flat training objective (i.e., equally treats all output tokens) and the hierarchical AMR structure, which limits the model generalization. To bridge this gap, we propose a Hierarchical Curriculum Learning (HCL) framework with Structure-level (SC) and Instance-level Curricula (IC). SC switches progressively from core to detail AMR semantic elements while IC transits from structure-simple to -complex AMR instances during training. Through these two warming-up processes, HCL reduces the difficulty of learning complex structures, thus the flat model can better adapt to the AMR hierarchy. Extensive experiments on AMR2.0, AMR3.0, structure-complex and out-of-distribution situations verify the effectiveness of HCL.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑