This paper contributes to the challenge of learning a function on streamed multimodal data through evaluation. The core of the result of our paper is the combination of two quite different approaches to this problem. One comes from the mathematically principled technology of signatures and log-signatures as representations for streamed data, while the other draws on the techniques of recurrent neural networks (RNN). The ability of the former to manage high sample rate streams and the latter to manage large scale nonlinear interactions allows hybrid algorithms that are easy to code, quicker to train, and of lower complexity for a given accuracy.
We illustrate the approach by approximating the unknown functional as a controlled differential equation. Linear functionals on solutions of controlled differential equations are the natural universal class of functions on data streams. Following this approach, we propose a hybrid Logsig-RNN algorithm that learns functionals on streamed data. By testing on various datasets, i.e. synthetic data, NTU RGB+D 120 skeletal action data, and Chalearn2013 gesture data, our algorithm achieves the outstanding accuracy with superior efficiency and robustness.
Exploiting Symmetric Temporally Sparse BPTT for Efficient RNN Training
Institute of Neuroinformatics, University of Zurich and ETH Zurich(苏黎世大学和苏黎世联邦理工学院神经信息学研究所);Department of Microelectronics, Delft University of Technology(代尔夫特理工大学微电子系)
Geometry-Induced Long-Range Correlations in Recurrent Neural Network Quantum States
University of Waterloo(滑铁卢大学);Perimeter Institute for Theoretical Physics(圆周理论物理研究所);NYU Tandon School of Engineering(纽约大学坦登工程学院);Vector Institute(向量研究所)