基于简单离散状态空间的时间序列有效建模
Effectively Modeling Time Series with Simple Discrete State Spaces
摘要:
时间序列建模是一个成熟的问题,它通常要求方法(1)表达复杂的依赖关系,(2)预测长期前景,(3)有效地训练长序列。状态空间模型(SSM)是时间序列的经典模型,之前的工作将SSM与深度学习层相结合,以实现高效的序列建模。然而,我们发现这些先前的方法的基本限制,证明他们的SSM表示不能表达自回归时间序列过程。因此,我们引入SpaceTime,一个新的状态空间时间序列架构,提高了所有三个标准。对于表现力,我们提出了一个新的SSM参数化的基础上的同伴矩阵-离散时间过程的规范表示-这使得SpaceTime的SSM层学习理想的自回归过程。对于长期预测,我们引入了一个“闭环”的同伴SSM的变化,这使得SpaceTime通过生成自己的逐层输入来预测许多未来的时间步长。为了有效的训练和推理,我们引入了一种算法,该算法减少了伴随矩阵的前向传递的内存和计算。有了序列长度$\ell$和状态空间大小$d$,我们就可以简单地从$\tilde{O}(d \ell)$到$\tilde{O}(d + \ell)$。在实验中,我们的贡献导致了广泛和多样化的基准测试的最先进的结果,6 / 7 ECG和语音时间序列分类的最佳或次佳AUROC,以及14 / 16 Informer预测任务的最佳MSE。此外,我们发现SpaceTime(1)适合AR($p$)过程,之前的深度SSM失败了,(2)在更长的时间范围内比现有技术更准确地预测,(3)比Transformers和LSTM在真实世界ETTh 1数据上的训练速度提高了73%和80%。
英文摘要:
Time series modeling is a well-established problem, which often requires that methods (1) expressively represent complicated dependencies, (2) forecast long horizons, and (3) efficiently train over long sequences. State-space models (SSMs) are classical models for time series, and prior works combine SSMs with deep learning layers for efficient sequence modeling. However, we find fundamental limitations with these prior approaches, proving their SSM representations cannot express autoregressive time series processes. We thus introduce SpaceTime, a new state-space time series architecture that improves all three criteria. For expressivity, we propose a new SSM parameterization based on the companion matrix -- a canonical representation for discrete-time processes -- which enables SpaceTime's SSM layers to learn desirable autoregressive processes. For long horizon forecasting, we introduce a "closed-loop" variation of the companion SSM, which enables SpaceTime to predict many future time-steps by generating its own layer-wise inputs. For efficient training and inference, we introduce an algorithm that reduces the memory and compute of a forward pass with the companion matrix. With sequence length $\ell$ and state-space size $d$, we go from $\tilde{O}(d \ell)$ naïvely to $\tilde{O}(d + \ell)$. In experiments, our contributions lead to state-of-the-art results on extensive and diverse benchmarks, with best or second-best AUROC on 6 / 7 ECG and speech time series classification, and best MSE on 14 / 16 Informer forecasting tasks. Furthermore, we find SpaceTime (1) fits AR($p$) processes that prior deep SSMs fail on, (2) forecasts notably more accurately on longer horizons than prior state-of-the-art, and (3) speeds up training on real-world ETTh1 data by 73% and 80% relative wall-clock time over Transformers and LSTMs.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。