arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.20503cs.LGcs.AI
收藏

一种用于长期时间序列预测的联邦大语言模型

A federated large language model for long-term time series forecasting

Concordia University(康考迪亚大学)

Raed Abdel-Sater, A. Ben Hamza

AI总结:

针对长期时间序列预测中的数据隐私与通信开销问题,提出联邦大语言模型 FedTime,通过 K-means 聚类、通道独立性和分块技术优化局部训练,在真实基准上显著提升预测性能并降低通信开销。

摘要:

集中式环境下的长期时间序列预测在数据隐私、通信开销和可扩展性方面带来了独特的挑战。为了应对这些挑战,我们提出了 FedTime,一种专为长程时间序列预测定制的联邦大语言模型(LLM)。具体而言,我们引入了带有微调和对齐策略的联邦预训练 LLM。在学习过程之前,我们采用 K-means 聚类将边缘设备或客户端划分为不同的簇,从而促进更有针对性的模型训练。我们还结合了通道独立性和分块(patching)以更好地保留局部语义信息,确保保留重要的上下文细节,同时将信息丢失的风险降至最低。我们通过在各种真实世界预测基准上的大量实验证明了 FedTime 模型的有效性,展示了相对于近期方法的显著改进。此外,我们证明了 FedTime 在简化资源使用方面的效率,从而降低了通信开销。

英文摘要:

Long-term time series forecasting in centralized environments poses unique challenges regarding data privacy, communication overhead, and scalability. To address these challenges, we propose FedTime, a federated large language model (LLM) tailored for long-range time series prediction. Specifically, we introduce a federated pre-trained LLM with fine-tuning and alignment strategies. Prior to the learning process, we employ K-means clustering to partition edge devices or clients into distinct clusters, thereby facilitating more focused model training. We also incorporate channel independence and patching to better preserve local semantic information, ensuring that important contextual details are retained while minimizing the risk of information loss. We demonstrate the effectiveness of our FedTime model through extensive experiments on various real-world forecasting benchmarks, showcasing substantial improvements over recent approaches. In addition, we demonstrate the efficiency of FedTime in streamlining resource usage, resulting in reduced communication overhead.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑