Multimodal time series forecasting is crucial in real-world applications, where decisions depend on both numerical data and contextual signals. The core challenge is to effectively combine temporal numerical patterns with the context embedded in other modalities, such as text. While most existing methods align textual features with time-series patterns one step at a time, they neglect the multiscale temporal influences of contextual information such as time-series cycles and dynamic shifts. This mismatch between local alignment and global textual context can be addressed by spectral decomposition, which separates time series into frequency components capturing both short-term changes and long-term trends. In this paper, we propose SpecTF, a simple yet effective framework that integrates the effect of textual data on time series in the frequency domain. Our method extracts textual embeddings, projects them into the frequency domain, and fuses them with the time series' spectral components using a lightweight cross-attention mechanism. This adaptively reweights frequency bands based on textual relevance before mapping the results back to the temporal domain for predictions. Experimental results demonstrate that SpecTF significantly outperforms state-of-the-art models across diverse multi-modal time series datasets while utilizing considerably fewer parameters. Code is available at https://github.com/hiepnh137/SpecTF.
Multimodal Physical Activity Forecasting in Free-Living Clinical Settings: Hunting Opportunities for Just-in-Time Interventions
College of Health Solutions, Arizona State University(亚利桑那州立大学健康解决方案学院);School of Computing and Augmented Intelligence, Arizona State University(亚利桑那州立大学计算与增强智能学院);Edson College of Nursing and Health Innovation, Arizona State University(亚利桑那州立大学埃德森护理与健康创新学院)
PA-RNet: Perturbation-Aware Residual Network for Robust Multimodal Time Series Forecasting
Institute of Computing Technology, Guangzhou University(广州大学计算机技术学院);School of Computer Science and Technology, Dalian University of Technology(大连理工大学计算机科学与技术学院)
SCENARIODIFF: A Scenario-level Guidance Framework for Multimodal Time Series Forecasting--Extended Version
VinUniversity;FPT Software AI Center, FPT Corporation(FPT软件AI中心,FPT集团);VNU University of Engineering and Technology(VNU工程技术大学);Aalborg University(奥尔堡大学)
Deep Information Fusion for Electric Vehicle Charging Station Occupancy Forecasting
L3S Research Center, Leibniz University Hannover(莱布尼茨汉诺威大学L3S研究中心);Data Science & Intelligent Systems (DSIS) Research Group, University of Bonn(波恩大学数据科学与智能系统(DSIS)研究组)
FuXi-Air: Urban Air Quality Forecasting Based on Emission-Meteorology-Pollutant multimodal Machine Learning
Shanghai Key Laboratory of Atmospheric Particle Pollution and Prevention(上海大气颗粒污染与防治重点实验室);National Observations and Research Station for Wetland Ecosystems of the Yangtze Estuary(长江入海口湿地生态系统观测与研究站);Department of Environmental Science and Engineering(环境科学与工程学院);Shanghai Academy of AI for Science (SAIS)(上海人工智能科学研究院);MOE Laboratory for National Development and Intelligent Governance(教育部国家发展与智能治理实验室);Shanghai Institute for Energy and Carbon Neutrality Strategy(上海能源与碳中和战略研究院);IRDR ICoE on Risk Interconnectivity and Governance on Weather/Climate Extremes Impact and Public Health(国际减灾与发展研究院风险互联与治理国际联合实验室);Shanghai Institute of Eco-Chongming (SIEC)(上海生态崇明研究院);Shanghai Environmental Monitoring Center (SEMC)(上海市环境监测中心)
Fidel-TS: A High-Fidelity Multimodal Benchmark for Time Series Forecasting
Department of Computer Science and Engineering(计算机科学与工程系);The Chinese University of Hong Kong(香港中文大学);School of Software(软件学院);Tsinghua University(清华大学);ZJU-UIUC Institute(浙大-UIUC研究院);Zhejiang University(浙江大学);School of Data Science(数据科学学院);The Chinese University of Hong Kong, Shenzhen(深圳香港中文大学)
UniDiff: A Unified Diffusion Framework for Multimodal Time Series Forecasting
School of Artificial Intelligence, OPtics and ElectroNics (iOPEN), Northwestern Polytechnical University, Xi’an 710072, China(人工智能学院、光学与电子学(iOPEN)、西北工业大学,西安710072,中国);Institute of Artificial Intelligence (TeleAI), China Telecom, China(人工智能研究所(TeleAI)、中国电信,中国)
Research Institute for Infrastructure Paradigm Shift, Yachiyo Engineering Co., Ltd.(基础设施范式转换研究所,八千代工程株式会社)
最近,由于气候变化引起的前所未有的天气条件,洪水灾害已成为一个社会问题。对暴雨立即作出反应对于减轻经济损失和迅速恢复都很重要。时空降水预报可以提高大坝入库流量预测的准确性,提前6小时以上减轻洪水损失。然而,由于目标预测值与地面实况值之间存在不可约的偏差,普通的ConvLSTM在实际降水预报中存在着可预报范围大于3个时间步的局限性。本文提出了一种雨码的时空降水代码到代码的预测方法。我们提出了一种新的雨的功能,表示一个时间雨的过程,使用多帧融合的时间步长减少。我们基于标准ConvLSTM对各种术语范围进行了雨码研究。我们应用了一个坝区内的日本雨季小时降水数据,2006年至2019年下约12.7万小时,每年5月至10月。我们将雷达分析每小时的数据应用于面积为136 x 148 km 2的中央更广泛的区域。最后,我们在几个预报范围内提供了雨码大小和每小时准确度之间的敏感性研究。
Multi-Modal Fusion of In-Situ Video Data and Process Parameters for Online Forecasting of Cookie Drying Readiness
Department of Mechanical Science and Engineering, University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校机械科学与工程系);Department of Mechanical Engineering, University of Michigan(密歇根大学机械工程系)
DyMRL: Dynamic Multispace Representation Learning for Multimodal Event Forecasting in Knowledge Graph
Huazhong University of Science and Technology(华中科技大学);Centre for Learning, Teaching and Technology(学习、教学与技术中心);The Education University of Hong Kong(香港教育大学)