arXiv 2305.188382023-05-30cs.LG

Client：用于多元长期时间序列预测的跨变量线性集成增强Transformer

Client: Cross-variable Linear Integrated Enhanced Transformer for Multivariate Long-Term Time Series Forecasting

Shanghai Jiaotong University（上海交通大学）；Eastern Institute for Advanced Study（东方理工高等研究院）；Hefei University of Technology（合肥工业大学）

Jiaxin Gao, Wenbo Hu, Yuntian Chen

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对多元长期时间序列预测中注意力模块捕获跨时间依赖性的质疑，提出Client模型，通过线性模块学习趋势、注意力模块捕获跨变量依赖，在九个数据集上以最少计算时间和内存消耗实现SOTA性能。

摘要：

长期时间序列预测（LTSF）是现代社会的一个关键方面，在促进长期规划和开发预警系统中发挥着重要作用。尽管最近已经为LTSF引入了许多基于Transformer的模型，但关于注意力模块在捕获跨时间依赖性方面的有效性已经出现了质疑。在本研究中，我们设计了一个掩码序列实验来验证这一假设，并随后提出了“用于多元长期时间序列预测的跨变量线性集成增强Transformer”（Client），这是一种先进的模型，其性能优于传统的基于Transformer的模型和线性模型。Client采用线性模块来学习趋势信息，并采用注意力模块来捕获跨变量依赖性。同时，它简化了嵌入和位置编码层，并用投影层替换了解码器模块。本质上，Client融合了非线性和跨变量依赖性，这使其区别于传统的线性模型和基于Transformer的模型。在九个真实世界数据集上的大量实验证实了Client的最先进（SOTA）性能，并且与之前的基于Transformer的模型相比，其计算时间和内存消耗最少。我们的代码可在https://github.com/daxin007/Client获取。

英文摘要：

Long-term time series forecasting (LTSF) is a crucial aspect of modern society, playing a pivotal role in facilitating long-term planning and developing early warning systems. While many Transformer-based models have recently been introduced for LTSF, a doubt have been raised regarding the effectiveness of attention modules in capturing cross-time dependencies. In this study, we design a mask-series experiment to validate this assumption and subsequently propose the "Cross-variable Linear Integrated ENhanced Transformer for Multivariate Long-Term Time Series Forecasting" (Client), an advanced model that outperforms both traditional Transformer-based models and linear models. Client employs linear modules to learn trend information and attention modules to capture cross-variable dependencies. Meanwhile, it simplifies the embedding and position encoding layers and replaces the decoder module with a projection layer. Essentially, Client incorporates non-linearity and cross-variable dependencies, which sets it apart from conventional linear models and Transformer-based models. Extensive experiments with nine real-world datasets have confirmed the SOTA performance of Client with the least computation time and memory consumption compared with the previous Transformer-based models. Our code is available at https://github.com/daxin007/Client.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。