arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2108.12844cs.CL
收藏

幕后:Few-Shot事件分类中触发偏差问题的探索

Behind the Scenes: An Exploration of Trigger Biases Problem in Few-Shot Event Classification

Peking University(北京大学)

Peiyi Wang, Runxin Xu, Tianyu Liu, Damai Dai, Baobao Chang, Zhifang Sui

摘要:

Few-Shot事件分类(FSEC)旨在开发一种用于事件预测的模型,该模型可以推广到具有有限数量的注释数据的新事件类型。现有的FSEC研究在不同的基准上都取得了很高的精度。然而,我们发现他们遭受的触发偏见,这意味着一些触发词和目标事件类型之间的统计同质性,我们总结为触发重叠和触发分离。偏差可能导致上下文绕过问题,即,可以通过仅查看触发词而忽略整个上下文来获得正确的分类。因此,现有的模型在推广到真实场景中看不见的数据方面可能很弱。为了进一步揭示模型的触发偏差和评估模型的泛化能力,我们提出了两种新的采样方法,触发均匀采样(TUS)和融合采样(COS),用于评估过程中的Meta任务构建。此外,为了解决FSEC模型中的上下文绕过问题,我们引入了对抗训练和触发器重构技术。实验表明,这些技术不仅有助于提高性能,而且还提高了模型的泛化能力。

英文摘要:

Few-Shot Event Classification (FSEC) aims at developing a model for event prediction, which can generalize to new event types with a limited number of annotated data. Existing FSEC studies have achieved high accuracy on different benchmarks. However, we find they suffer from trigger biases that signify the statistical homogeneity between some trigger words and target event types, which we summarize as trigger overlapping and trigger separability. The biases can result in context-bypassing problem, i.e., correct classifications can be gained by looking at only the trigger words while ignoring the entire context. Therefore, existing models can be weak in generalizing to unseen data in real scenarios. To further uncover the trigger biases and assess the generalization ability of the models, we propose two new sampling methods, Trigger-Uniform Sampling (TUS) and COnfusion Sampling (COS), for the meta tasks construction during evaluation. Besides, to cope with the context-bypassing problem in FSEC models, we introduce adversarial training and trigger reconstruction techniques. Experiments show these techniques help not only improve the performance, but also enhance the generalization ability of models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑