arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2410.20439cs.LGcs.AIstat.ML
收藏

TEAFormers:用于多维时间序列预测的张量增强 Transformer

TEAFormers: TEnsor-Augmented Transformers for Multi-Dimensional Time Series Forecasting

New York University(纽约大学);University of California, Riverside(加州大学河滨分校);Notre Dame University(圣母大学)

Linghang Kong, Elynn Chen, Yuzhou Chen, Yuefeng Han

AI总结:

本文提出 TEAFormer,通过可兼容现有 Transformer 的 TEA 张量展开与压缩模块保留多维时序结构,在三个基准和三种模型上显著提升预测性能并降低计算量。

摘要:

多维时间序列数据(如矩阵和张量变量时间序列)在经济学、金融学和气候科学等领域日益普遍。传统 Transformer 模型虽然擅长处理序列数据,却无法有效保留这些多维结构,因为其内部运算实际上会将多维观测展平为向量,从而丢失关键的多维关系与模式。为解决这一问题,我们提出张量增强 Transformer(Tensor-Augmented Transformer,TEAFormer)这一新方法,在 Transformer 框架内引入张量展开与压缩,以维持并利用固有的多维结构,从而降低计算成本并提高预测精度。TEAFormer 的核心特征是张量增强(Tensor-Augmentation,TEA)模块,它利用张量展开增强多视图特征学习,并通过张量压缩实现高效信息聚合和降低计算负载。TEA 模块并非只是一种特定模型架构,而是一个通用组件,与 Transformer 的注意力机制和编码器—解码器结构高度兼容,因此可适配现有 Transformer 架构。我们将 TEA 模块集成到三种流行的时间序列 Transformer 模型中,并在三个真实世界基准上进行了全面实验,结果显示性能显著提升,凸显了 TEAFormers 在前沿时间序列预测中的潜力。

英文摘要:

Multi-dimensional time series data, such as matrix and tensor-variate time series, are increasingly prevalent in fields such as economics, finance, and climate science. Traditional Transformer models, though adept with sequential data, do not effectively preserve these multi-dimensional structures, as their internal operations in effect flatten multi-dimensional observations into vectors, thereby losing critical multi-dimensional relationships and patterns. To address this, we introduce the Tensor-Augmented Transformer (TEAFormer), a novel method that incorporates tensor expansion and compression within the Transformer framework to maintain and leverage the inherent multi-dimensional structures, thus reducing computational costs and improving prediction accuracy. The core feature of the TEAFormer, the Tensor-Augmentation (TEA) module, utilizes tensor expansion to enhance multi-view feature learning and tensor compression for efficient information aggregation and reduced computational load. The TEA module is not just a specific model architecture but a versatile component that is highly compatible with the attention mechanism and the encoder-decoder structure of Transformers, making it adaptable to existing Transformer architectures. Our comprehensive experiments, which integrate the TEA module into three popular time series Transformer models across three real-world benchmarks, show significant performance enhancements, highlighting the potential of TEAFormers for cutting-edge time series forecasting.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑