Sensorformer：具有全局 patch 压缩的跨 patch 注意力对高维多变量时间序列预测是有效的
Sensorformer: Cross-patch attention with global-patch compression is effective for high-dimensional multivariate time series forecasting
AI总结：
针对多变量时间序列中跨时间与跨变量依赖难以融合的问题，提出 Sensorformer，通过全局 patch 压缩和跨 patch 注意力同时建模两类依赖，降低复杂度并在 9 个数据集上验证优越性能。
摘要：
在现有的基于 Transformer 的多变量时间序列预测方法中，将每个变量序列视为一个 token 且仅显式提取跨变量依赖关系的 iTransformer，以及采用通道独立策略且仅显式提取跨时间依赖关系的 PatchTST，二者的表现都显著优于大多数同时提取跨时间和跨变量依赖关系的 Channel-Dependent Transformer。这表明现有的基于 Transformer 的多变量时间序列预测方法仍难以有效融合这两类信息。我们将该问题归因于不同变量之间因果关系中的动态时间滞后。因此，我们提出了一种新的多变量时间序列预测 Transformer——Sensorformer，它首先压缩全局 patch 信息，然后从压缩后的表示中同时提取跨变量和跨时间依赖关系。即使在变量之间存在动态因果滞后的情况下，Sensorformer 也能有效捕捉正确的变量间相关性和因果关系，同时将纯跨 patch 自注意力的计算复杂度从 $O(D^2 \cdot Patch\\_num^2 \cdot d\\_model)$ 降低到 $O(D^2 \cdot Patch\\_num \cdot d\\_model)$。在 9 个主流真实世界多变量时间序列预测数据集上进行的大量对比实验和消融实验证明了 Sensorformer 的优越性。遵循 Time-series-library 风格的 Sensorformer 实现以及用于复现主要结果的脚本已公开发布于 https://github.com/BigYellowTiger/Sensorformer。
英文摘要：
Among the existing Transformer-based multivariate time series forecasting methods, iTransformer, which treats each variable sequence as a token and only explicitly extracts cross-variable dependencies, and PatchTST, which adopts a channel-independent strategy and only explicitly extracts cross-time dependencies, both significantly outperform most Channel-Dependent Transformer that simultaneously extract cross-time and cross-variable dependencies. This indicates that existing Transformer-based multivariate time series forecasting methods still struggle to effectively fuse these two types of information. We attribute this issue to the dynamic time lags in the causal relationships between different variables. Therefore, we propose a new multivariate time series forecasting Transformer, Sensorformer, which first compresses the global patch information and then simultaneously extracts cross-variable and cross-time dependencies from the compressed representations. Sensorformer can effectively capture the correct inter-variable correlations and causal relationships, even in the presence of dynamic causal lags between variables, while also reducing the computational complexity of pure cross-patch self-attention from $O(D^2 \cdot Patch\_num^2 \cdot d\_model)$ to $O(D^2 \cdot Patch\_num \cdot d\_model)$. Extensive comparative and ablation experiments on 9 mainstream real-world multivariate time series forecasting datasets demonstrate the superiority of Sensorformer. The implementation of Sensorformer, following the style of the Time-series-library and scripts for reproducing the main results, is publicly available at https://github.com/BigYellowTiger/Sensorformer
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
