arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2410.01183cs.CLstat.CO
收藏

FastLexRank:用于结构化社交媒体帖子的高效词汇排序

FastLexRank: Efficient Lexical Ranking for Structuring Social Media Posts

Institute for Social Research(社会研究所);University of Michigan(密歇根大学)

Mao Li, Frederick Conrad, Johann Gagnon-Bartsch

AI总结:

本文提出 FastLexRank,通过优化句子图平稳分布计算将 LexRank 的时空复杂度由 O(n²) 降至 O(n),在保持评分一致的同时支持大规模社交媒体语料的实时核心推文识别。

摘要:

我们提出 FastLexRank(脚注:https://github.com/LiMaoUM/FastLexRank),这是 LexRank 文本排序算法的一种高效且可扩展的实现。为解决原始 LexRank 方法的计算和内存复杂度问题而设计,FastLexRank 在不损害结果质量或准确性的情况下,将时间和内存需求从 O(n²) 显著降低至 O(n)。通过采用优化方法计算句子图的平稳分布,FastLexRank 在提高计算效率的同时,保持与原始 LexRank 评分完全相同的结果。本文详细介绍了使其能够实时处理社交媒体语料库等大型数据集的算法改进。实证结果证明了其有效性,我们提议将其用于识别核心推文,随后可利用先进 NLP 技术作进一步分析。FastLexRank 为文本中心性计算提供了可扩展方案,满足了对数字内容高效处理日益增长的需求。

英文摘要:

We present FastLexRank\footnote{https://github.com/LiMaoUM/FastLexRank}, an efficient and scalable implementation of the LexRank algorithm for text ranking. Designed to address the computational and memory complexities of the original LexRank method, FastLexRank significantly reduces time and memory requirements from $\mathcal{O}(n^2)$ to $\mathcal{O}(n)$ without compromising the quality or accuracy of the results. By employing an optimized approach to calculating the stationary distribution of sentence graphs, FastLexRank maintains an identical results with the original LexRank scores while enhancing computational efficiency. This paper details the algorithmic improvements that enable the processing of large datasets, such as social media corpora, in real-time. Empirical results demonstrate its effectiveness, and we propose its use in identifying central tweets, which can be further analyzed using advanced NLP techniques. FastLexRank offers a scalable solution for text centrality calculation, addressing the growing need for efficient processing of digital content.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑