arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1007.3254cs.CLcond-mat.stat-mechphysics.soc-ph
收藏

区分事实与虚构:基于复杂网络的文本模式识别

Distinguishing Fact from Fiction: Pattern Recognition in Texts Using Complex Networks

Colorado School of Mines(科罗拉多矿业学院)

J. T. Stevanak, David M. Larue, Lincoln D. Carr

摘要:

我们建立了具体的数学标准来区分不同类型的书面故事,虚构和非虚构。具体来说,我们从小说和新闻故事中构建了一个语义网络,其中$N$独立词作为顶点或节点,边或链接分配给给定顶点的$m$位置内出现的词;我们称之为$m$词距离。然后,我们使用复杂网络理论的措施来区分新闻和小说,研究所需的最小文本长度以及优化的词距离$m$。文献样本被发现是最有效的表示其相应的幂律度分布$P(k)$和聚类系数$C(k)$ ;我们还研究了平均测地距离,并发现我们所有的文本是小世界网络。我们在$k=\sqrt{N}$观察到一个自然的断点,在那里度分布中的幂律发生了变化,导致$P(k)$的主体和尾部分别符合幂律。我们的线性判别分析产生了$73.8 \pm 5.15%$的准确性,正确分类的小说和$69.1 \pm 1.22%$的新闻故事。我们发现最佳的词距离为$m=4$,最小文本长度为100到200个词$N$。

英文摘要:

We establish concrete mathematical criteria to distinguish between different kinds of written storytelling, fictional and non-fictional. Specifically, we constructed a semantic network from both novels and news stories, with $N$ independent words as vertices or nodes, and edges or links allotted to words occurring within $m$ places of a given vertex; we call $m$ the word distance. We then used measures from complex network theory to distinguish between news and fiction, studying the minimal text length needed as well as the optimized word distance $m$. The literature samples were found to be most effectively represented by their corresponding power laws over degree distribution $P(k)$ and clustering coefficient $C(k)$; we also studied the mean geodesic distance, and found all our texts were small-world networks. We observed a natural break-point at $k=\sqrt{N}$ where the power law in the degree distribution changed, leading to separate power law fit for the bulk and the tail of $P(k)$. Our linear discriminant analysis yielded a $73.8 \pm 5.15%$ accuracy for the correct classification of novels and $69.1 \pm 1.22%$ for news stories. We found an optimal word distance of $m=4$ and a minimum text length of 100 to 200 words $N$.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑