近来，人们越来越关注将预训练大语言模型（LLM）用于各种时间序列应用。然而，LLM通过预训练建立的语义空间仍未得到充分探索，而该空间可能有助于产生更具区分性和信息量的表示，从而促进时间序列预测。为此，我们提出了利用LLM的语义空间启发提示学习方法（Semantic Space Informed Prompt learning with LLM，$S^2$IP-LLM），将预训练语义空间与时间序列嵌入空间对齐，并基于从联合空间中学习到的提示执行时间序列预测。我们首先设计了一个专为跨模态对齐定制的分词模块，该模块显式拼接分解后的时间序列成分的补丁，以创建有效编码时间动态的嵌入。接下来，我们利用预训练词元嵌入推导语义锚点，并通过最大化联合空间中的余弦相似度，将选定的锚点与时间序列嵌入对齐。这样，$S^2$IP-LLM可以检索相关语义锚点作为提示，为表现出不同时间动态的时间序列提供强指标（上下文）。通过在多个基准数据集上的充分实证研究，我们证明所提出的$S^2$IP-LLM能够取得优于最先进基线的预测性能。此外，我们的消融研究和可视化验证了由语义空间启发的提示学习的必要性。
英文摘要：
Recently, there has been a growing interest in leveraging pre-trained large language models (LLMs) for various time series applications. However, the semantic space of LLMs, established through the pre-training, is still underexplored and may help yield more distinctive and informative representations to facilitate time series forecasting. To this end, we propose Semantic Space Informed Prompt learning with LLM ($S^2$IP-LLM) to align the pre-trained semantic space with time series embeddings space and perform time series forecasting based on learned prompts from the joint space. We first design a tokenization module tailored for cross-modality alignment, which explicitly concatenates patches of decomposed time series components to create embeddings that effectively encode the temporal dynamics. Next, we leverage the pre-trained word token embeddings to derive semantic anchors and align selected anchors with time series embeddings by maximizing the cosine similarity in the joint space. This way, $S^2$IP-LLM can retrieve relevant semantic anchors as prompts to provide strong indicators (context) for time series that exhibit different temporal dynamics. With thorough empirical studies on multiple benchmark datasets, we demonstrate that the proposed $S^2$IP-LLM can achieve superior forecasting performance over state-of-the-art baselines. Furthermore, our ablation studies and visualizations verify the necessity of prompt learning informed by semantic space.
Monash University（莫纳什大学）；La Trobe University（乐卓博大学）；Shanghai Jiao Tong University（上海交通大学）；East China Normal University（华东师范大学）；Griffith University（格里菲斯大学）
Multivariate time series forecasting has long received significant attention in real-world applications, such as energy consumption and traffic prediction. While recent methods demonstrate good forecasting abilities, they have three fundamental limitations. (i) Discrete neural architectures: Interlacing individually parameterized spatial and temporal blocks to encode rich underlying patterns leads to discontinuous latent state trajectories and higher forecasting numerical errors. (ii) High complexity: Discrete approaches complicate models with dedicated designs and redundant parameters, leadin…
Heterogeneity-Informed Meta-Parameter Learning for Spatiotemporal Time Series Forecasting
Southern University of Science and Technology（南方科技大学）；The University of Tokyo（东京大学）；Hong Kong University of Science and Technology（香港科技大学）；Squirrel AI（松鼠AI）；Jilin University（吉林大学）
Process-Informed Forecasting of Complex Thermal Dynamics in Pharmaceutical Manufacturing
Department of Agricultural and Environmental Sciences, University of Bari（巴里大学农业与环境科学系）；Department of Mechanical, Mathematics, and Management (DMMM), Polytechnic University of Bari（巴里理工学院机械、数学与管理系）；Division of Applied Mathematics, Brown University（布朗大学应用数学系）
Beyond Extrapolation: Knowledge Utilization Paradigm with Bidirectional Inspiration for Time Series Forecasting
College of Computer Science, Sichuan University（四川大学计算机科学学院）；Department of Computer and Information Science, University of Macau（澳门大学计算机与信息科学系）；School of Information and Communication Engineering, University of Electronic Science and Technology of China（电子科技大学信息与通信工程学院）
Nonstationary Time Series Forecasting via Unknown Distribution Adaptation
Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；Guangdong University（广东大学）；Carnegie Mellon University（卡内基梅隆大学）；Shantou University（汕头大学）
SPAT: Sensitivity-based Multihead-attention Pruning on Time Series Forecasting Models
School of Artificial Intelligence, Nanjing University, China（人工智能学院，南京大学）；National Key Laboratory for Novel Software Technology, Nanjing University, China（新型软件技术国家重点实验室，南京大学）；School of Electronic Science and Engineering, Nanjing University, China（电子科学与工程学院，南京大学）
本文提出 SPAT 结构化剪枝方法，利用动态 SEND 指标识别并移除整个冗余注意力模块，在多元时间序列预测中降低误差与 FLOPs，并超越多种 SOTA 方法。
Recent studies have shown that deep learning models such as RNNs and Transformers have brought significant performance gains for long-term forecasting of time series because they effectively utilize historical information. We found, however, that there is still great room for improvement in how to preserve historical information in neural networks while avoiding overfitting to noise presented in the history. Addressing this allows better utilization of the capabilities of deep learning models. To this end, we design a \textbf{F}requency \textbf{i}mproved \textbf{L}egendre \textbf{M}emory model…
Addressing Concept Shift in Online Time Series Forecasting: Detect-then-Adapt
Chinese Academy of Sciences (CASIA)（中国科学院）；University of Chinese Academy of Sciences (UCAS)（中国科学院大学）；Alibaba Group（阿里巴巴集团）；Squirrel AI Group（松鼠AI智适应教育集团）；Meta AI Group（元宇宙人工智能集团）
A Market-Rule-Informed Neural Network for Efficient Imbalance Electricity Price Forecasting
Delft University of Technology（代尔夫特理工大学）；Austrian Institute of Technology（奥地利技术研究院）；London Business School（伦敦商学院）；University of Technology Sydney（悉尼大学）；The Chinese University of Hongkong (Shenzhen)（香港中文大学（深圳））
vLinear: A Powerful Linear Model for Multivariate Time Series Forecasting
State Key Laboratory of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University（人工智能通用基础理论国家重点实验室，智能科学与技术学院，北京大学）
Modeling Temporal Dependencies within the Target for Long-Term Time Series Forecasting
School of Computing and Artificial Intelligence, Engineering Research Center of Sustainable Urban Intelligent Transportation, Ministry of Education, National Engineering Laboratory of Integrated Transportation Big Data Application Technology, Manufacturing Industry Chains Collaboration and Information Support Technology Key Laboratory of Sichuan Province, Southwest Jiaotong University, Chengdu 611756, China（计算机学院与人工智能学院，可持续城市智能交通工程研究中心，教育部，集成交通运输大数据应用技术国家工程实验室，四川省制造业链条协同与信息支撑技术重点实验室，西南交通大学，成都611756，中国）；School of Computing, The University of Leeds, LS2 9JT Leeds, U.K.（计算机学院，利兹大学，利兹，英国）
Grey-informed neural network for time-series forecasting
School of Communication, Qufu Normal University（曲阜师范大学传媒学院）；School of Computer Science and Information Engineering, Chuzhou Univeristy（滁州学院计算机科学与信息工程学院）
PINN-Cast: Exploring the Role of Continuous-Depth NODE in Transformers and Physics Informed Loss as Soft Physical Constraints in Short-term Weather Forecasting
From Samples to Scenarios: A New Paradigm for Probabilistic Forecasting
ZJU-UIUC Institute, Zhejiang University（浙江大学ZJU-UIUC研究院）；Department of Computer Science and Engineering, The Chinese University of Hong Kong（香港中文大学计算机科学与工程系）；School of Software, Tsinghua University（清华大学软件学院）
Physics-informed deep learning for infectious disease forecasting
School of Chemical, Materials, and Biomedical Engineering, University of Georgia（化学、材料与生物医学工程学院，佐治亚大学）；Odum School of Ecology and Center for the Ecology of Infectious Diseases, University of Georgia（生态学院与传染病生态研究中心，佐治亚大学）；School of Environmental, Civil Agricultural and Mechanical Engineering, College of Engineering, University of Georgia（环境、土木农业与机械工程学院，工程学院，佐治亚大学）；Department of Epidemiology & Biostatistics, University of Georgia（流行病学与生物统计学系，佐治亚大学）；Department of Infectious Diseases, College of Veterinary Medicine, University of Georgia（传染病系，兽医学学院，佐治亚大学）；Center for Influenza Disease and Emergence Research, University of Georgia（流感疾病与新兴病研究中心，佐治亚大学）
Benchmarking Physics-Informed Time-Series Models for Operational Global Station Weather Forecasting
Department of Computer Science and Engineering, Hong Kong University of Science and Technology, Hong Kong SAR China（香港科技大学计算机科学与工程系）；Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, China（南方科技大学计算机科学与工程系）；School of Computer and Information Sciences, University of Newcastle, Newcastle, Australia（新castle大学计算机与信息科学学院）；Hangzhou Innovation Institute of Beihang University, Hangzhou, China（北京航空航天大学杭州创新研究院）；Shanghai Artificial Intelligence Laboratory, Shanghai, China（上海人工智能实验室）
Differential-Integral Neural Operator for Long-Term Turbulence Forecasting
Tsinghua University（清华大学）；University of Science and Technology of China（中国科学技术大学）；The Chinese University of Hong Kong（香港中文大学）；Nanyang Technological University（南洋理工大学）；Tencent（腾讯）
Integrating Spatiotemporal Features in LSTM for Spatially Informed COVID-19 Hospitalization Forecasting
Heilbrunn Department of Population and Family Health, Columbia University Mailman School of Public Health（哥伦比亚大学流行病学与家庭健康系）；Department of Geography, University of Colorado（科罗拉多大学地理系）；Population Council（人口理事会）
Florida Atlantic University（佛罗里达大西洋大学）；Harbor Branch Oceanographic Institute, Florida Atlantic University（佛罗里达大西洋大学港口分支海洋研究所）；Georgia Institute of Technology（佐治亚理工学院）；Washington University School of Medicine（华盛顿大学医学院）
Ocean current, fluid mechanics, and many other spatio-temporal physical dynamical systems are essential components of the universe. One key characteristic of such systems is that certain physics laws -- represented as ordinary/partial differential equations (ODEs/PDEs) -- largely dominate the whole process, irrespective of time or location. Physics-informed learning has recently emerged to learn physics for accurate prediction, but they often lack a mechanism to leverage localized spatial and temporal correlation or rely on hard-coded physics parameters. In this paper, we advocate a physics-co…
Interpretable Physics-Informed Load Forecasting for U.S. Grid Resilience: SHAP-Guided Ensemble Validation in Hybrid Deep Learning Under Extreme Weather
1Dept. of Computer Science, Midwestern State University, Dallas, TX, USA 2O\&M Analytics, AES Clean Energy, The AES Corporation, Louisville, CO, USA 3Dept. of Information；Communication Engineering, Pabna University of Science
