arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1708.00065cs.LG
收藏

神经元事件序列预测的时间依赖表示

Time-Dependent Representation for Neural Event Sequence Prediction

Google AI(谷歌人工智能);Google Brain(谷歌大脑)

Yang Li, Nan Du, Samy Bengio

摘要:

现有的序列预测方法大多关注时间无关的序列,其中事件之间的实际时间跨度是无关的,事件之间的距离仅仅是它们在序列中的顺序位置之间的差异。虽然序列的这种与时间无关的视图适用于诸如自然语言的数据,处理句子中的单词,对于在自然出现时在不相等间隔的时间点观察和收集的许多真实世界事件来说,这是不合适的和低效的,例如,当一个人去杂货店或打电话时。事件之间的时间跨度可以携带关于人类行为的序列依赖性的重要信息。在这项工作中,我们提出了一套使用时间序列预测的方法。由于RNN等神经序列模型更适合处理令牌式输入,因此我们提出了两种用于时间相关事件表示的方法,这是基于对时间在日常生活中如何令牌化的直觉以及之前关于嵌入上下文化的工作。我们还介绍了两种使用下一个事件持续时间作为训练序列预测模型的正则化的方法。我们讨论这些方法的基础上,递归神经网络。我们评估这些方法以及基线模型的五个数据集,类似于各种序列预测任务。实验表明,所提出的方法提供了准确性增益在一系列设置的基线模型。

英文摘要:

Existing sequence prediction methods are mostly concerned with time-independent sequences, in which the actual time span between events is irrelevant and the distance between events is simply the difference between their order positions in the sequence. While this time-independent view of sequences is applicable for data such as natural languages, e.g., dealing with words in a sentence, it is inappropriate and inefficient for many real world events that are observed and collected at unequally spaced points of time as they naturally arise, e.g., when a person goes to a grocery store or makes a phone call. The time span between events can carry important information about the sequence dependence of human behaviors. In this work, we propose a set of methods for using time in sequence prediction. Because neural sequence models such as RNN are more amenable for handling token-like input, we propose two methods for time-dependent event representation, based on the intuition on how time is tokenized in everyday life and previous work on embedding contextualization. We also introduce two methods for using next event duration as regularization for training a sequence prediction model. We discuss these methods based on recurrent neural nets. We evaluate these methods as well as baseline models on five datasets that resemble a variety of sequence prediction tasks. The experiments revealed that the proposed methods offer accuracy gain over baseline models in a range of settings.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑