arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2302.10570cs.CLcs.AIcs.SC
收藏

基于Transformer和HowNet义素知识融合的协同驱动语义一致性识别

Co-Driven Recognition of Semantic Consistency via the Fusion of Transformer and HowNet Sememes Knowledge

Huazhong University of Science and Technology(华中科技大学)

Fan Chen, Yan Huang, Xinfang Zhang, Kang Luo, Jinxuan Zhu, Ruixian He

摘要:

语义一致性识别的目的是检测和判断两个文本句子的语义是否一致。然而,现有的方法通常会遇到同义词、一词多义和难以理解长文本的挑战。针对上述问题,提出一种基于Transformer和知网语义知识融合的协同驱动语义一致性识别方法。首先利用Transformer对句子内部结构进行数据驱动的多级编码,然后引入语义知识库HowNet进行知识驱动,对句子对之间的语义知识关联进行建模。然后,利用软注意力,将知识与语义矩阵相融合,进行交互式注意力计算。最后,利用双向长短期记忆网络(BiLSTM)对概念语义信息进行编码并推断语义一致性。在两个金融文本匹配数据集(BQ,AFQMC)和一个跨语言对抗数据集(PAWSX)上进行了释义识别实验。与DSSM、MwAN、DRCN等轻量级模型和ERNIE等预训练模型相比,该模型不仅有效地提高了语义一致性识别的准确率(在BQ数据集上与DSSM、MWAN和DRCN模型相比分别提高了2.19%、5.57%和6.51%),而且减少了模型参数的数量(约为16M)。此外,在知网语义知识的驱动下,该方法有望适应长文本的场景。

英文摘要:

Semantic consistency recognition aims to detect and judge whether the semantics of two text sentences are consistent with each other. However, the existing methods usually encounter the challenges of synonyms, polysemy and difficulty to understand long text. To solve the above problems, this paper proposes a co-driven semantic consistency recognition method based on the fusion of Transformer and HowNet sememes knowledge. Multi-level encoding of internal sentence structures via data-driven is carried out firstly by Transformer, sememes knowledge base HowNet is introduced for knowledge-driven to model the semantic knowledge association among sentence pairs. Then, interactive attention calculation is carried out utilizing soft-attention and fusion the knowledge with sememes matrix. Finally, bidirectional long short-term memory network (BiLSTM) is exploited to encode the conceptual semantic information and infer the semantic consistency. Experiments are conducted on two financial text matching datasets (BQ, AFQMC) and a cross-lingual adversarial dataset (PAWSX) for paraphrase identification. Compared with lightweight models including DSSM, MwAN, DRCN, and pre-training models such as ERNIE etc., the proposed model can not only improve the accuracy of semantic consistency recognition effectively (by 2.19%, 5.57% and 6.51% compared with the DSSM, MWAN and DRCN models on the BQ dataset), but also reduce the number of model parameters (to about 16M). In addition, driven by the HowNet sememes knowledge, the proposed method is promising to adapt to scenarios with long text.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑