arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2410.16489cs.LG
收藏

LLM-TS Integrator:整合大语言模型以增强时间序列建模

LLM-TS Integrator: Integrating LLM for Enhanced Time Series Modeling

Borealis AI(博雷亚利斯人工智能公司)

Can Chen, Gabriel Oliveira, Hossein Sharifi Noghabi, Tristan Sylvain

AI总结:

本文提出LLM-TS Integrator框架,通过互信息模块桥接传统时间序列模型与LLM表示,并利用样本重加权模块动态优化双重损失权重,在多个时间序列任务上取得领先性能。

摘要:

时间序列(TS)建模在天气预测和异常检测等动态系统中至关重要。近期研究利用大语言模型(LLMs)进行时间序列建模,借助其强大的模式识别能力。这些方法主要将LLMs定位为预测主干,往往忽略了传统时间序列模型中的数学建模,例如周期性。然而,忽视LLMs的潜力也意味着忽视了它们的模式识别能力。为弥补这一不足,我们提出了LLM-TS Integrator,一种将LLMs的能力有效整合到传统时间序列建模中的新型框架。该整合的核心是我们的互信息模块。此互信息模块的核心是一个传统时间序列模型,并通过LLM衍生的见解加以增强,以提升预测能力。这种增强通过最大化传统模型的时间序列表示与LLM的文本表示对应物之间的互信息来实现,从而桥接两种模态。此外,我们认识到样本在传统预测损失和互信息最大化这两个损失中的重要性各不相同。为应对这种差异性,我们引入了样本重加权模块以改善信息利用。该模块为每个样本分配双重权重:一个用于预测损失,另一个用于互信息损失,并通过双层优化动态优化这些权重。我们的方法在五个主流时间序列任务上取得了最先进或可比的性能,包括短期和长期预测、插补、分类和异常检测。

英文摘要:

Time series~(TS) modeling is essential in dynamic systems like weather prediction and anomaly detection. Recent studies utilize Large Language Models (LLMs) for TS modeling, leveraging their powerful pattern recognition capabilities. These methods primarily position LLMs as the predictive backbone, often omitting the mathematical modeling within traditional TS models, such as periodicity. However, disregarding the potential of LLMs also overlooks their pattern recognition capabilities. To address this gap, we introduce \textit{LLM-TS Integrator}, a novel framework that effectively integrates the capabilities of LLMs into traditional TS modeling. Central to this integration is our \textit{mutual information} module. The core of this \textit{mutual information} module is a traditional TS model enhanced with LLM-derived insights for improved predictive abilities. This enhancement is achieved by maximizing the mutual information between traditional model's TS representations and LLM's textual representation counterparts, bridging the two modalities. Moreover, we recognize that samples vary in importance for two losses: traditional prediction and mutual information maximization. To address this variability, we introduce the \textit{sample reweighting} module to improve information utilization. This module assigns dual weights to each sample: one for prediction loss and another for mutual information loss, dynamically optimizing these weights via bi-level optimization. Our method achieves state-of-the-art or comparable performance across five mainstream TS tasks, including short-term and long-term forecasting, imputation, classification, and anomaly detection.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑