FiCoTS: Fine-to-Coarse LLM-Enhanced Hierarchical Cross-Modality Interaction for Time Series Forecasting
School of Advanced Interdisciplinary Sciences, University of Chinese Academy Sciences(中国科学院大学先进交叉学科学院);MAIS, Institute of Automation, Chinese Academy of Science(中国科学院自动化研究所MAIS);Great Bay University(大亚大学)
Yafei Lyu, Hao Zhou, Lu Zhang, Xu Yang, Zhiyong Liu
Time series forecasting is central to data analysis and web technologies. The recent success of Large Language Models (LLMs) offers significant potential for this field, especially from the cross-modality aspect. Most methods adopt an LLM-as-Predictor paradigm, using LLM as the forecasting backbone and designing modality alignment mechanisms to enable LLM to understand time series data. However, the semantic information in the two modalities of time series and text differs significantly, making it challenging for LLM to fully understand time series data. To mitigate this challenge, our work follows an LLM-as-Enhancer paradigm to fully utilize the advantage of LLM in text understanding, where LLM is only used to encode text modality to complement time series modality. Based on this paradigm, we propose FiCoTS, an LLM-enhanced fine-to-coarse framework for multimodal time series forecasting. Specifically, the framework facilitates progressive cross-modality interaction by three levels in a fine-to-coarse scheme: First, in the token-level modality alignment module, a dynamic heterogeneous graph is constructed to filter noise and align time series patches with text tokens; Second, in the feature-level modality interaction module, a global cross-attention mechanism is introduced to enable each time series variable to connect with relevant textual contexts; Third, in the decision-level modality fusion module, we design a gated network to adaptively fuse the results of the two modalities for robust predictions. These three modules work synergistically to let the two modalities interact comprehensively across three semantic levels, enabling textual information to effectively support temporal prediction. Extensive experiments on seven real-world benchmarks demonstrate that our model achieves state-of-the-art performance. The codes will be released publicly.
DSFNet: Learning Dual-Domain Spectral Operators for Multi-Modality Spatio-Temporal Forecasting in Urban Transportation Systems
Southeast University(东南大学);Jiangsu Province Collaborative Innovation Center of Modern Urban Traffic Technologies(江苏省现代城市交通技术协同创新中心);City University of Hong Kong(香港城市大学);School of Mathematics, Southeast University(东南大学数学学院);Systems Research Institute of the Polish Academy of Sciences(波兰科学院系统研究所);Luoyang Normal University(洛阳师范学院);Purple Mountain Laboratories(紫金山实验室);AGH University of Krakow(AGH科技大学)
Multi-Modal Fusion of In-Situ Video Data and Process Parameters for Online Forecasting of Cookie Drying Readiness
Department of Mechanical Science and Engineering, University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校机械科学与工程系);Department of Mechanical Engineering, University of Michigan(密歇根大学机械工程系)
Multi-Source Urban Traffic Flow Forecasting with Drone and Loop Detector Data
Urban Transport Systems Laboratory (LUTS), École Polytechnique Fédérale de Lausanne (EPFL)(城市交通系统实验室(LUTS),瑞士联邦理工学院(EPFL));Visual Intelligence for Transportation Laboratory (VITA), École Polytechnique Fédérale de Lausanne (EPFL)(交通视觉智能实验室(VITA),瑞士联邦理工学院(EPFL))
MM-ISTS: Cooperating Irregularly Sampled Time Series Forecasting with Multimodal Vision-Text LLMs
East China Normal University Shanghai China;Centre for Artificial Intelligence;Robotics, Hong Kong Institute of Science \& Innovation, Chinese Academy of Sciences Hong Kong China;Department of Computing, The Hong Kong Polytechnic University Hong Kong China;East China Normal University;Robotics, Hong Kong Institute of Science \& Innovation, Chinese Academy of Sciences;Department of Computing, The Hong Kong Polytechnic University
Advancing Real-time Pandemic Forecasting Using Large Language Models: A COVID-19 Case Study
Johns Hopkins University(约翰斯·霍普金斯大学);Mila - Quebec AI Institute(米拉-魁北克人工智能研究所);Harvard T.H. Chan School of Public Health(哈佛陈曾熙公共卫生学院);Harvard University(哈佛大学);Universite de Montréal(蒙特利尔大学);Duke University(杜克大学);Johns Hopkins Bloomberg School of Public Health(约翰斯·霍普金斯大学布隆伯格公共卫生学院)
LeMoLE: LLM-Enhanced Mixture of Linear Experts for Time Series Forecasting
Hong Kong University of Science and Technology(香港科技大学);Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));University of Cologne(科隆大学)
MM-FinEval: A Multi-Task Multimodal Benchmark for Real-World Financial Forecasting
Northwestern University(西北大学);New Jersey Institute of Technology(新泽西理工学院);Georgia Institute of Technology(佐治亚理工学院);Pace University(佩斯大学);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳))
Time-FFM: Towards LM-Empowered Federated Foundation Model for Time Series Forecasting
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));National University of Singapore(新加坡国立大学);Salesforce AI Research(Salesforce AI研究部);Squirrel AI(松鼠AI)
UniDiff: A Unified Diffusion Framework for Multimodal Time Series Forecasting
School of Artificial Intelligence, OPtics and ElectroNics (iOPEN), Northwestern Polytechnical University, Xi’an 710072, China(人工智能学院、光学与电子学(iOPEN)、西北工业大学,西安710072,中国);Institute of Artificial Intelligence (TeleAI), China Telecom, China(人工智能研究所(TeleAI)、中国电信,中国)
DrFuse: Learning Disentangled Representation for Clinical Multi-Modal Fusion with Missing Modality and Modal Inconsistency
The Hong Kong Polytechnic University(香港理工大学);Hong Kong Baptist University(香港浸会大学);Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院)
InA-Probe: Instruction-Aware Active Probing for Time Series Forecasting with LLMs
Nanyang Technological University(南洋理工大学);Khalifa University(哈利法大学);Nanjing University of Aeronautics and Astronautics(南京航空航天大学);Singapore University of Technology and Design(新加坡科技设计大学)