arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.14978cs.AI
收藏

TS-HTFA:通过与大语言模型的分层无文本对齐推进时间序列预测

TS-HTFA: Advancing Time Series Forecasting via Hierarchical Text-Free Alignment with Large Language Models

School of Computer Science and Technology, East China Normal University(华东师范大学计算机科学与技术学院)

Pengfei Wang, Huanran Zheng, Qi'ao Xu, Silong Dai, Yiqiao Wang, Wenjing Yue, Wei Zhu, Tianwen Qian, Xiaoling Wang

AI总结:

针对LLM时间序列预测依赖成对文本且模态对齐不足的问题,本文提出TS-HTFA,以QR分解虚拟文本和可学习提示替代文本,并在输入、特征、输出三层实现无文本跨模态对齐,取得SOTA效果。

摘要:

鉴于大语言模型(LLM)在序列建模方面的巨大潜力,新兴研究已开始将其应用于时间序列预测。尽管取得了显著进展,现有方法仍面临两个关键挑战:1)依赖大量成对文本数据,限制了模型的适用性;2)文本与时间序列之间存在显著的模态鸿沟,导致对齐不足和性能欠佳。本文提出分层无文本对齐(TS-HTFA)这一新方法,利用分层对齐充分挖掘LLM的表征能力,同时消除对文本数据的依赖。具体而言,我们用基于QR分解词嵌入和可学习提示的自适应虚拟文本替代成对文本数据。此外,我们在输入、特征和输出三个层级建立全面的跨模态对齐。在多个时间序列基准上的大量实验表明,HTFA达到了最先进性能,显著提升了预测准确率和泛化能力。

英文摘要:

Given the significant potential of large language models (LLMs) in sequence modeling, emerging studies have begun applying them to time-series forecasting. Despite notable progress, existing methods still face two critical challenges: 1) their reliance on large amounts of paired text data, limiting the model applicability, and 2) a substantial modality gap between text and time series, leading to insufficient alignment and suboptimal performance. In this paper, we introduce \textbf{H}ierarchical \textbf{T}ext-\textbf{F}ree \textbf{A}lignment (\textbf{TS-HTFA}), a novel method that leverages hierarchical alignment to fully exploit the representation capacity of LLMs while eliminating the dependence on text data. Specifically, we replace paired text data with adaptive virtual text based on QR decomposition word embeddings and learnable prompt. Furthermore, we establish comprehensive cross-modal alignment at three levels: input, feature, and output. Extensive experiments on multiple time-series benchmarks demonstrate that HTFA achieves state-of-the-art performance, significantly improving prediction accuracy and generalization.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑