arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2310.03243stat.MLcs.AIcs.LG
收藏

面向时间序列数据的稀疏深度学习:理论与应用

Sparse Deep Learning for Time Series Data: Theory and Applications

Purdue University(普渡大学);University of Pennsylvania(宾夕法尼亚大学)

Mingxuan Zhang, Yan Sun, Faming Liang

AI总结:

针对现有稀疏深度学习多聚焦独立同分布数据、对依赖数据研究不足的问题,本文研究了时间序列等依赖数据下的稀疏深度学习理论,证明稀疏RNN可一致估计且预测渐近正态,在不确定性量化、自回归阶数识别和模型压缩上均优于现有方法,具备多领域应用价值。

摘要:

稀疏深度学习已成为提升深度神经网络在不确定性量化、变量选择和大规模网络压缩等领域性能的常用技术。但现有研究大多聚焦于观测值独立同分布(i.i.d.)的问题,针对时间序列数据、自然语言处理中的序列数据等观测值存在依赖的问题研究较少。本文旨在通过研究依赖数据下的稀疏深度学习理论填补这一空白。我们证明,在适当假设下,稀疏循环神经网络(RNNs)可被一致估计,其预测结果服从渐近正态分布,从而能够正确量化预测不确定性。数值实验结果表明,在时间序列数据的预测不确定性量化任务上,稀疏深度学习的表现优于共形预测等现有最优方法。此外,结果显示所提方法能够一致识别时间序列数据的自回归阶数,且在大规模模型压缩方面的表现优于现有方法。该方法在金融、医疗、能源等同时关注精确点估计和预测不确定性量化的领域具有重要的实际应用价值。

英文摘要:

Sparse deep learning has become a popular technique for improving the performance of deep neural networks in areas such as uncertainty quantification, variable selection, and large-scale network compression. However, most existing research has focused on problems where the observations are independent and identically distributed (i.i.d.), and there has been little work on the problems where the observations are dependent, such as time series data and sequential data in natural language processing. This paper aims to address this gap by studying the theory for sparse deep learning with dependent data. We show that sparse recurrent neural networks (RNNs) can be consistently estimated, and their predictions are asymptotically normally distributed under appropriate assumptions, enabling the prediction uncertainty to be correctly quantified. Our numerical results show that sparse deep learning outperforms state-of-the-art methods, such as conformal predictions, in prediction uncertainty quantification for time series data. Furthermore, our results indicate that the proposed method can consistently identify the autoregressive order for time series data and outperform existing methods in large-scale model compression. Our proposed method has important practical implications in fields such as finance, healthcare, and energy, where both accurate point estimates and prediction uncertainty quantification are of concern.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑