arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1909.04985cs.CLcs.LGstat.ML
收藏

基于时态语言模型的动态作者表示学习

Learning Dynamic Author Representations with Temporal Language Models

Sorbonne Université(索邦大学);Facebook AI Research(脸书人工智能研究院)

Edouard Delasalles, Sylvain Lamprier, Ludovic Denoyer

摘要:

语言模型是许多工作的核心,特别是在文本挖掘和信息检索社区。这些统计模型旨在提取单词分布,从简单的单字模型到使用潜变量的递归方法,这些方法可以捕获文本中的微妙依赖关系。然而,这些模型只从单词序列中学习,很少考虑作者的身份以及出版日期。我们提出了一个神经模型,基于循环语言建模,其目的是捕捉语言扩散趋势,作者社区随着时间的推移。通过用作者和时间向量状态来调节语言模型,我们能够利用文本上下文之间的潜在依赖关系。这使我们能够在两个真实世界的语料库上击败几个时间和非时间语言基线,并学习随时间变化的有意义的作者表示。

英文摘要:

Language models are at the heart of numerous works, notably in the text mining and information retrieval communities. These statistical models aim at extracting word distributions, from simple unigram models to recurrent approaches with latent variables that capture subtle dependencies in texts. However, those models are learned from word sequences only, and authors' identities, as well as publication dates, are seldom considered. We propose a neural model, based on recurrent language modeling, which aims at capturing language diffusion tendencies in author communities through time. By conditioning language models with author and temporal vector states, we are able to leverage the latent dependencies between the text contexts. This allows us to beat several temporal and non-temporal language baselines on two real-world corpora, and to learn meaningful author representations that vary through time.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑