NeST：面向基于大语言模型的时间序列预测的邻域感知语义对齐与时间调制
NeST: Neighborhood-aware semantic alignment and temporal modulation for LLM based time series forecasting
AI总结：
NeST框架通过邻域感知语义对齐与时间调制构建文本整合提示词微调LLM，在8个基准及9个真实光伏数据集的时间序列预测中，MSE、R²等指标优于现有方法，泛化性强。
摘要：
将在离散文本数据上训练的大语言模型（LLM）适配到连续时间序列信号预测任务颇具挑战性。尽管对LLM进行微调可实现此类适配，但在提示词中有效整合文本与时间序列信息至关重要。现有基于LLM的时间序列预测方法多通过简单拼接或参数繁重的交叉注意力融合两种模态；此外，现有方法采用分解技术嵌入时间序列数据，可能无法充分捕捉复杂的时间动态。为解决这些局限，我们提出基于邻域感知语义对齐与时间调制的框架NeST，以构建用于微调LLM的新型文本整合时间序列提示词。首先，生成邻域感知文本原型，该原型经优化后可表示LLM预训练词标记嵌入的局部邻域；其次，通过最近邻对比目标将其与时间序列输入的时间表示对齐，之后检索出top-k个最相关的文本原型；最后，引入文本原型条件时间调制，利用检索到的文本原型自适应缩放和平移时间序列特征。大量实验表明，NeST在8个基准数据集上始终优于现有最优方法，长期预测的均方误差（MSE）平均降低1.2%；此外，其展现出强泛化能力，零样本预测的MSE平均降低4.9%。除基准数据集外，NeST在9个真实世界分布式光伏功率预测任务数据集上也表现稳健，平均决定系数（R²）提升3.3%。这些发现证明了NeST在将LLM适配到各类时间序列预测任务上的有效性与可推广性。
英文摘要：
Adapting Large Language Models (LLMs) trained on discrete text data, to forecast continuous time series signals is challenging. While finetuning the LLMs enables such adaptation, effectively integrating both textual and time series information in the prompt is critical. Current LLM-based time series forecasting methods combine the two modalities through simple concatenation or parameter heavy cross-attention. Moreover, existing methods embed time series data using decomposition techniques that may inadequately capture complex temporal dynamics. To address these limitations, we propose neighborhood-aware semantic alignment and temporal modulation based framework (NEST) to formulate a new text-integrated time series prompt to finetune the LLM. First, we generate neighborhood-aware text prototypes that are optimized to represent local neighborhoods of pretrained word token embeddings of the LLM. Second, we align them with temporal representations of the time series input using a nearest-neighbor contrastive objective, after which the top-k most relevant text prototypes are retrieved. Third, we introduce text prototype conditioned temporal modulation that uses the retrieved text prototypes to adaptively scale and shift time series features. Extensive experiments demonstrate that NeST consistently outperforms state-of-the-art methods across eight benchmarks, achieving an average 1.2\% reduction in MSE for long-term forecasting. In addition, it demonstrates strong generalization, yielding an average 4.9\% reduction in MSE in zero-shot forecasting. Beyond benchmark datasets, NeST also delivers robust performance on a real-world distributed photovoltaic power forecasting task across nine datasets, improving the average R$^2$ score by 3.3\%. These findings demonstrate the effectiveness and generalizability of NeST for adapting LLMs to diverse time series forecasting tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
