arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1606.08689cs.CLcs.IR
收藏

用于流文档及其内容联合表示的层次神经语言模型

Hierarchical Neural Language Models for Joint Representation of Streaming Documents and their Content

Yahoo Labs(雅虎实验室);University of Southern California(南加州大学)

Nemanja Djuric, Hao Wu, Vladan Radosavljevic, Mihajlo Grbovic, Narayan Bhamidipati

摘要:

我们考虑的问题,学习分布式表示的文件数据流。这些文档被表示为低维向量,并使用具有两个嵌入式神经语言模型的分层框架与单词标记的分布式向量表示联合学习。特别是,我们利用流中的文档的上下文,并使用其中一个语言模型来建模文档序列,另一个模型中的单词序列。模型学习单词标记和文档的连续向量表示,使得语义相似的文档和单词在公共向量空间中接近。我们讨论扩展我们的模型,它可以被应用到个性化推荐和社会关系挖掘,通过添加进一步的用户层的层次结构,从而学习用户特定的向量来表示个人偏好。我们在MovieLens的公共电影评级数据集上验证了所学习的表示,以及在雅虎服务器上收集的包括三个月用户活动日志的大规模雅虎新闻数据上验证了所学习的表示。结果表明,该模型可以学习文档和单词标记的有用表示,性能大大优于当前最先进的。

英文摘要:

We consider the problem of learning distributed representations for documents in data streams. The documents are represented as low-dimensional vectors and are jointly learned with distributed vector representations of word tokens using a hierarchical framework with two embedded neural language models. In particular, we exploit the context of documents in streams and use one of the language models to model the document sequences, and the other to model word sequences within them. The models learn continuous vector representations for both word tokens and documents such that semantically similar documents and words are close in a common vector space. We discuss extensions to our model, which can be applied to personalized recommendation and social relationship mining by adding further user layers to the hierarchy, thus learning user-specific vectors to represent individual preferences. We validated the learned representations on a public movie rating data set from MovieLens, as well as on a large-scale Yahoo News data comprising three months of user activity logs collected on Yahoo servers. The results indicate that the proposed model can learn useful representations of both documents and word tokens, outperforming the current state-of-the-art by a large margin.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑