时间依赖观测的深度神经网络的理论分析
Theoretical analysis of deep neural networks for temporally dependent observations
摘要:
深度神经网络是用非线性模式对随时间推移的观察进行建模的强大工具。尽管神经网络在这样的环境中被广泛使用,但深度神经网络的大多数理论发展都是在独立观测的假设下进行的,并且时间依赖观测的理论结果很少。为了弥补这一差距,我们研究了深度神经网络对非线性时间序列数据建模的理论特性。具体而言,在混合型假设下,建立了具有ReLU激活函数的(稀疏)前馈神经网络的预测误差的非渐近界。这些假设是温和的,使得它们包括广泛的时间序列模型,包括自回归模型。与独立观测值相比,建立的收敛速率具有额外的对数因子,以补偿由于数据点之间的依赖性而导致的额外复杂性。通过各种数值模拟设置以及应用到宏观经济数据集的理论结果的支持。
英文摘要:
Deep neural networks are powerful tools to model observations over time with non-linear patterns. Despite the widespread use of neural networks in such settings, most theoretical developments of deep neural networks are under the assumption of independent observations, and theoretical results for temporally dependent observations are scarce. To bridge this gap, we study theoretical properties of deep neural networks on modeling non-linear time series data. Specifically, non-asymptotic bounds for prediction error of (sparse) feed-forward neural network with ReLU activation function is established under mixing-type assumptions. These assumptions are mild such that they include a wide range of time series models including auto-regressive models. Compared to independent observations, established convergence rates have additional logarithmic factors to compensate for additional complexity due to dependence among data points. The theoretical results are supported via various numerical simulation settings as well as an application to a macroeconomic data set.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。