arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2205.04012cs.CL
收藏

使用负聚焦预训练改进否定检测

Improving negation detection with negation-focused pre-training

The University of Melbourne(墨尔本大学);RMIT University(皇家墨尔本理工大学);MBZUAI(穆罕默德·本·扎耶德人工智能大学)

Thinh Hung Truong, Timothy Baldwin, Trevor Cohn, Karin Verspoor

摘要:

否定是一种常见的语言特征,在许多语言理解任务中起着至关重要的作用,但由于否定在不同类型的语篇中的表现形式不同,它仍然是一个难题。最近的工作表明,最先进的NLP模型在各种任务中对包含否定的样本表现不佳,并且否定检测模型不能很好地跨域传输。我们提出了一种新的以否定为中心的预训练策略,包括有针对性的数据增强和否定掩蔽,以更好地将否定信息纳入语言模型。在常见基准上进行的大量实验表明,我们提出的方法在强基线NegBERT上提高了否定检测性能和泛化能力(Khandewal和Sawant,2020)。

英文摘要:

Negation is a common linguistic feature that is crucial in many language understanding tasks, yet it remains a hard problem due to diversity in its expression in different types of text. Recent work has shown that state-of-the-art NLP models underperform on samples containing negation in various tasks, and that negation detection models do not transfer well across domains. We propose a new negation-focused pre-training strategy, involving targeted data augmentation and negation masking, to better incorporate negation information into language models. Extensive experiments on common benchmarks show that our proposed approach improves negation detection performance and generalizability over the strong baseline NegBERT (Khandewal and Sawant, 2020).

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑