arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1704.07463cs.CL
收藏

基于节省空间算法的流式词嵌入

Streaming Word Embeddings with the Space-Saving Algorithm

Johns Hopkins University(约翰斯·霍普金斯大学);Denison University(丹尼森大学)

Chandler May, Kevin Duh, Benjamin Van Durme, Ashwin Lall

摘要:

我们开发了一个流(一次通过,有界内存)词嵌入算法的基础上,规范的跳跃与负采样算法在word2vec。我们比较我们的流媒体算法word2vec经验,通过测量每个算法下的单词对之间的余弦相似性,并通过应用每个算法在Twitter样本流的两个月间隔的主题标签预测的下游任务。然后,我们讨论了这些实验的结果,得出结论,他们提供了部分验证我们的方法作为一个流替代word2vec。最后,我们讨论了潜在的故障模式,并建议今后的工作方向。

英文摘要:

We develop a streaming (one-pass, bounded-memory) word embedding algorithm based on the canonical skip-gram with negative sampling algorithm implemented in word2vec. We compare our streaming algorithm to word2vec empirically by measuring the cosine similarity between word pairs under each algorithm and by applying each algorithm in the downstream task of hashtag prediction on a two-month interval of the Twitter sample stream. We then discuss the results of these experiments, concluding they provide partial validation of our approach as a streaming replacement for word2vec. Finally, we discuss potential failure modes and suggest directions for future work.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑