arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2103.03446cs.CL
收藏

基于渐进式自监督注意学习的情感情感分析模型

Enhanced Aspect-Based Sentiment Analysis Models with Progressive Self-supervised Attention Learning

Xiamen University(厦门大学);Institute of Software, Chinese Academy of Sciences(中国科学院软件研究所);University of Chinese Academy of Sciences(中国科学院大学);University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);University of Rochester(罗切斯特大学);Tianjin University(天津大学)

Jinsong Su, Jialong Tang, Hui Jiang, Ziyao Lu, Yubin Ge, Linfeng Song, Deyi Xiong, Le Sun, Jiebo Luo

摘要:

在基于方面的情感分析(ABSA)中,许多神经模型都配备了注意力机制,以量化每个上下文单词对情感预测的贡献。然而,这样的机制有一个缺点:只有少数频繁的话,情感极性往往被考虑到最终的情感决定,而丰富的不频繁的情感词被模型忽略。为了解决这个问题,我们提出了一个渐进的自监督注意力学习方法的注意力ABSA模型。在这种方法中,我们迭代地对所有训练实例进行情感预测,同时不断学习有用的注意力监督信息。在训练期间,在每次迭代中,基于其注意力权重或梯度识别的对情感预测具有最高影响的上下文单词被提取为对每个实例的正确/不正确预测具有积极/误导影响的单词。以这种方式提取的单词被屏蔽以用于后续迭代。为了利用这些提取的单词来改进ABSA模型,我们用正则化项来增强传统的训练目标,该正则化项鼓励ABSA模型不仅充分利用提取的活跃上下文单词,而且还降低了那些误导性单词的权重。我们将所提出的方法集成到三个国家的最先进的神经ABSA模型。实验结果和深入的分析表明,我们的方法产生了更好的注意力的结果,显着提高了所有三个模型的性能。我们在https://github.com/DeepLearnXMU/PSSAttention上发布源代码和训练模型。

英文摘要:

In aspect-based sentiment analysis (ABSA), many neural models are equipped with an attention mechanism to quantify the contribution of each context word to sentiment prediction. However, such a mechanism suffers from one drawback: only a few frequent words with sentiment polarities are tended to be taken into consideration for final sentiment decision while abundant infrequent sentiment words are ignored by models. To deal with this issue, we propose a progressive self-supervised attention learning approach for attentional ABSA models. In this approach, we iteratively perform sentiment prediction on all training instances, and continually learn useful attention supervision information in the meantime. During training, at each iteration, context words with the highest impact on sentiment prediction, identified based on their attention weights or gradients, are extracted as words with active/misleading influence on the correct/incorrect prediction for each instance. Words extracted in this way are masked for subsequent iterations. To exploit these extracted words for refining ABSA models, we augment the conventional training objective with a regularization term that encourages ABSA models to not only take full advantage of the extracted active context words but also decrease the weights of those misleading words. We integrate the proposed approach into three state-of-the-art neural ABSA models. Experiment results and in-depth analyses show that our approach yields better attention results and significantly enhances the performance of all three models. We release the source code and trained models at https://github.com/DeepLearnXMU/PSSAttention.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑