arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1611.04822cs.CL
收藏

SimDoc:基于主题序列比对的文档相似性框架

SimDoc: Topic Sequence Alignment based Document Similarity Framework

University of Bonn(波恩大学);Emory University(埃默里大学);Rygbee Inc.(Rygbee公司)

Gaurav Maheshwari, Priyansh Trivedi, Harshita Sahijwani, Kunal Jha, Sourish Dasgupta, Jens Lehmann

摘要:

文档相似性是估计给定文档对具有相似语义内容的程度的问题。一个准确的文档相似性度量可以改善一些企业相关的任务,如文档聚类,文本挖掘和问答。在本文中,我们表明,文档的主题流(通常被词袋技术忽视)在估计其相似性方面至关重要。为此,我们提出了一种新的语义文档相似性框架,称为SimDoc。我们的模型文件的主题序列,主题代表潜在的生成集群的相关词。然后,我们使用序列比对算法来估计它们的语义相似度。我们进一步概念化一种新的机制来计算主题主题相似度,以微调我们的系统。在我们的实验中,我们表明,SimDoc优于许多当代的词袋技术,准确计算文档相似性,并在实际应用中,如文档聚类。

英文摘要:

Document similarity is the problem of estimating the degree to which a given pair of documents has similar semantic content. An accurate document similarity measure can improve several enterprise relevant tasks such as document clustering, text mining, and question-answering. In this paper, we show that a document's thematic flow, which is often disregarded by bag-of-word techniques, is pivotal in estimating their similarity. To this end, we propose a novel semantic document similarity framework, called SimDoc. We model documents as topic-sequences, where topics represent latent generative clusters of related words. Then, we use a sequence alignment algorithm to estimate their semantic similarity. We further conceptualize a novel mechanism to compute topic-topic similarity to fine tune our system. In our experiments, we show that SimDoc outperforms many contemporary bag-of-words techniques in accurately computing document similarity, and on practical applications such as document clustering.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑