使用预训练大语言模型进行多变量时间序列预测
Using Pre-trained LLMs for Multivariate Time Series Forecasting
AI总结：
本研究利用预训练LLMs的跨领域知识迁移能力，提出多变量分块策略将时间序列特征嵌入仅解码器预训练Transformer，用于多变量需求时间序列预测，效果达当前最优水平，并通过权重诊断验证结论。
摘要：
预训练大语言模型（LLMs）蕴含大量知识，且训练需要巨量计算资源。我们利用这一资源，结合LLMs能够将知识与性能从一个领域甚至模态迁移到另一个看似无关领域的特性，来助力多变量需求时间序列预测任务。基于Transformer的方法中的注意力机制需要有值得关注的内容——而不仅仅是时间序列的样本。我们探索了将多变量输入时间序列映射到LLM词元嵌入空间的不同方法。具体而言，我们提出的新型多变量分块策略可将时间序列特征嵌入到仅解码器的预训练Transformer中，其取得的结果与当前最先进的时间序列预测模型相当。我们还使用了最新开发的基于权重的诊断方法来验证我们的发现。
英文摘要：
Pre-trained Large Language Models (LLMs) encapsulate large amounts of knowledge and take enormous amounts of compute to train. We make use of this resource, together with the observation that LLMs are able to transfer knowledge and performance from one domain or even modality to another seemingly-unrelated area, to help with multivariate demand time series forecasting. Attention in transformer-based methods requires something worth attending to -- more than just samples of a time-series. We explore different methods to map multivariate input time series into the LLM token embedding space. In particular, our novel multivariate patching strategy to embed time series features into decoder-only pre-trained Transformers produces results competitive with state-of-the-art time series forecasting models. We also use recently-developed weight-based diagnostics to validate our findings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
