arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2401.03591cs.CL
收藏

基于知识图谱与改进注意力机制的文本分类

Text Classification Based on Knowledge Graphs and Improved Attention Mechanism

Chongqing Jiaotong University(重庆交通大学)

Siyu Li, Lu Chen, Chenwei Song, Xinyi Liu

AI总结:

针对文本语义歧义问题,提出融合知识图谱与改进局部自注意力机制的Bi-GRU文本分类模型,通过信息增益选词、概念加权降噪优化分类,在多个数据集上取得有效准确率提升。

摘要:

为解决文本中的语义歧义问题,我们提出一种创新性结合知识图谱与改进注意力机制的模型。该模型利用现有知识库,用相关上下文概念丰富文本表示,同时在字符和词两个层级运行,通过融合这些概念深化对文本的理解。我们首先采用信息增益法筛选重要词汇,随后使用编码器-解码器框架对文本及相关概念进行编码。局部注意力机制会调整每个概念的权重,降低分类过程中无关或噪声概念的影响;我们改进了局部自注意力机制中注意力分数的计算公式,确保在文本中出现频率不同的词能获得更高的注意力分数。最后,模型采用双向门控循环单元(Bi-GRU),该单元在文本特征提取方面表现优异,可提升分类准确率。我们在AGNews、Ohsumed和TagMyNews等数据集上验证了模型性能,分别取得75.1%、58.7%和68.5%的准确率,证明了其在分类任务中的有效性。

英文摘要:

To resolve the semantic ambiguity in texts, we propose a model, which innovatively combines a knowledge graph with an improved attention mechanism. An existing knowledge base is utilized to enrich the text with relevant contextual concepts. The model operates at both character and word levels to deepen its understanding by integrating the concepts. We first adopt information gain to select import words. Then an encoder-decoder framework is used to encode the text along with the related concepts. The local attention mechanism adjusts the weight of each concept, reducing the influence of irrelevant or noisy concepts during classification. We improve the calculation formula for attention scores in the local self-attention mechanism, ensuring that words with different frequencies of occurrence in the text receive higher attention scores. Finally, the model employs a Bi-directional Gated Recurrent Unit (Bi-GRU), which is effective in feature extraction from texts for improved classification accuracy. Its performance is demonstrated on datasets such as AGNews, Ohsumed, and TagMyNews, achieving accuracy of 75.1%, 58.7%, and 68.5% respectively, showing its effectiveness in classifying tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑