多模态数据的抽象:多时间尺度回归模型的实证研究
Toward Abstraction from Multi-modal Data: Empirical Studies on Multiple Time-scale Recurrent Models
摘要:
多模态序列的抽象任务是具有挑战性的,因为它们需要更深层次的语义理解和新的文本生成的数据。虽然递归神经网络(RNN)可以用来对时间序列的上下文进行建模,但在大多数情况下,多模态数据的长期依赖性使得RNN通过时间训练的反向传播在时域中趋于消失。近年来,受多时间尺度递归神经网络(MTRNN)的启发,人们提出了一种门控递归单元(GRU)的扩展,称为多时间尺度门控递归单元(MTGRU),用于学习自然语言处理中的长期依赖关系。特别是,它还能够完成段落的抽象任务,因为时间常数定义得很好。在本文中,我们比较了MTRNN和MTGRU的学习性能以及它们在更高层次上的抽象表示(具有较慢的神经激活)。这是通过基于较小的数据集(来自非线性函数的二维时间序列)和相对较大的数据集(来自具有多模态数据的iCub操作任务的43维时间序列)进行两项研究来完成的。我们的结论是,门控递归机制可能是必要的学习长期的依赖关系,在大尺寸的多模态数据集(例如机器人操作的学习),即使自然语言命令是不涉及。但对于具有简单时间序列的较小学习任务,通用版本的递归模型(如MTRNN)足以完成抽象任务。
英文摘要:
The abstraction tasks are challenging for multi- modal sequences as they require a deeper semantic understanding and a novel text generation for the data. Although the recurrent neural networks (RNN) can be used to model the context of the time-sequences, in most cases the long-term dependencies of multi-modal data make the back-propagation through time training of RNN tend to vanish in the time domain. Recently, inspired from Multiple Time-scale Recurrent Neural Network (MTRNN), an extension of Gated Recurrent Unit (GRU), called Multiple Time-scale Gated Recurrent Unit (MTGRU), has been proposed to learn the long-term dependencies in natural language processing. Particularly it is also able to accomplish the abstraction task for paragraphs given that the time constants are well defined. In this paper, we compare the MTRNN and MTGRU in terms of its learning performances as well as their abstraction representation on higher level (with a slower neural activation). This was done by conducting two studies based on a smaller data- set (two-dimension time sequences from non-linear functions) and a relatively large data-set (43-dimension time sequences from iCub manipulation tasks with multi-modal data). We conclude that gated recurrent mechanisms may be necessary for learning long-term dependencies in large dimension multi-modal data-sets (e.g. learning of robot manipulation), even when natural language commands was not involved. But for smaller learning tasks with simple time-sequences, generic version of recurrent models, such as MTRNN, were sufficient to accomplish the abstraction task.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。