arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2002.09673cs.CL
收藏

基于自适应门注意的文本分类全局有效信息提取

Incorporating Effective Global Information via Adaptive Gate Attention for Text Classification

Ant Financial Services Group(蚂蚁金融服务集团);City University of Hong Kong(香港城市大学);Lingnan University(岭南大学);Hong Kong Polytechnic University(香港理工大学)

Xianming Li, Zongxi Li, Yingbin Zhao, Haoran Xie, Qing Li

摘要:

主流的文本分类研究集中在仅使用文本实例或引入外部知识(例如,手工制作特征和领域专家知识)。相比之下,一些语料库级的统计特征,如词频和分布,没有得到很好的利用。我们的工作表明,这种简单的统计信息可以有效地提高分类性能和显着的几个基线模型相比。本文提出了一种基于全局信息的自适应门注意模型(AGA+GI),该模型将全局统计特征与潜在语义特征相结合,注意层捕捉句子内部的依赖关系。为了缓解过拟合问题,我们提出了一种新的漏失机制,以提高泛化能力和性能稳定性。我们的实验表明,所提出的方法可以实现更好的准确性比基于CNN和基于RNN的方法没有全局信息的几个基准。

英文摘要:

The dominant text classification studies focus on training classifiers using textual instances only or introducing external knowledge (e.g., hand-craft features and domain expert knowledge). In contrast, some corpus-level statistical features, like word frequency and distribution, are not well exploited. Our work shows that such simple statistical information can enhance classification performance both efficiently and significantly compared with several baseline models. In this paper, we propose a classifier with gate mechanism named Adaptive Gate Attention model with Global Information (AGA+GI), in which the adaptive gate mechanism incorporates global statistical features into latent semantic features and the attention layer captures dependency relationship within the sentence. To alleviate the overfitting issue, we propose a novel Leaky Dropout mechanism to improve generalization ability and performance stability. Our experiments show that the proposed method can achieve better accuracy than CNN-based and RNN-based approaches without global information on several benchmarks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑