DeepSITH：通过跨时间尺度分解什么和什么时间的有效学习
DeepSITH: Efficient Learning via Decomposition of What and When Across Time Scales
摘要：
在一系列尺度上提取时间关系是人类感知和认知的标志-因此它是应用于现实世界问题的机器学习的关键特征。神经网络要么受到递归神经网络（RNN）中的梯度爆炸/消失问题的困扰，要么必须调整它们的参数来学习相关的时间尺度（例如，在LSTM中）。本文介绍了DeepSITH，这是一种由生物启发的尺度不变时间历史（SITH）模块串联而成的网络，层与层之间具有密集的连接。SITH模块用一组几何间隔的时间常数来响应它们的输入，使DeepSITH网络能够沿着连续的时间尺度来学习问题。我们将DeepSITH与LSTM和其他最近的RNN在几个时间序列预测和解码任务上进行了比较。DeepSITH在这些问题上实现了最先进的性能。
英文摘要：
Extracting temporal relationships over a range of scales is a hallmark of human perception and cognition -- and thus it is a critical feature of machine learning applied to real-world problems. Neural networks are either plagued by the exploding/vanishing gradient problem in recurrent neural networks (RNNs) or must adjust their parameters to learn the relevant time scales (e.g., in LSTMs). This paper introduces DeepSITH, a network comprising biologically-inspired Scale-Invariant Temporal History (SITH) modules in series with dense connections between layers. SITH modules respond to their inputs with a geometrically-spaced set of time constants, enabling the DeepSITH network to learn problems along a continuum of time-scales. We compare DeepSITH to LSTMs and other recent RNNs on several time series prediction and decoding tasks. DeepSITH achieves state-of-the-art performance on these problems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
