arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1902.08050cs.CL
收藏

基于知识驱动注意力的深度短文本分类

Deep Short Text Classification with Knowledge Powered Attention

Fudan University(复旦大学);CETC Big Data Research Institute Co.,Ltd.(中国电子科技集团大数据研究院有限公司);Shanghai Institute of Intelligent Electronics & Systems(上海智能电子与系统研究院);Shuyan Technology(书颜科技);Alibaba Group(阿里巴巴集团)

Jindong Chen, Yizhou Hu, Jingping Liu, Yanghua Xiao, Haiyun Jiang

摘要:

短文本分类是自然语言处理的重要任务之一。与段落或文档不同,短文本由于没有足够的上下文信息而更加模糊,这对分类提出了很大的挑战。在本文中,我们从外部知识源检索知识,以增强短文本的语义表示。我们将概念信息作为一种知识,并将其纳入深度神经网络。为了度量知识的重要性,我们引入了注意力机制,提出了基于知识驱动注意力的深度短文本分类(STCKA)。我们利用面向短文本的概念(C-ST)注意力和面向概念集的概念(C-CS)注意力从两个方面获取概念的权重。并在概念信息的帮助下对短文进行分类。与传统方法不同,我们的模型就像一个人一样,具有基于观察做出决策的内在能力(即,机器的训练数据),并更多地关注重要知识。我们还针对不同的任务在四个公共数据集上进行了广泛的实验。实验结果和案例研究表明,我们的模型优于国家的最先进的方法,证明了知识驱动的注意力的有效性。

英文摘要:

Short text classification is one of important tasks in Natural Language Processing (NLP). Unlike paragraphs or documents, short texts are more ambiguous since they have not enough contextual information, which poses a great challenge for classification. In this paper, we retrieve knowledge from external knowledge source to enhance the semantic representation of short texts. We take conceptual information as a kind of knowledge and incorporate it into deep neural networks. For the purpose of measuring the importance of knowledge, we introduce attention mechanisms and propose deep Short Text Classification with Knowledge powered Attention (STCKA). We utilize Concept towards Short Text (C- ST) attention and Concept towards Concept Set (C-CS) attention to acquire the weight of concepts from two aspects. And we classify a short text with the help of conceptual information. Unlike traditional approaches, our model acts like a human being who has intrinsic ability to make decisions based on observation (i.e., training data for machines) and pays more attention to important knowledge. We also conduct extensive experiments on four public datasets for different tasks. The experimental results and case studies show that our model outperforms the state-of-the-art methods, justifying the effectiveness of knowledge powered attention.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑