This paper describes a novel machine learning (ML) framework for tropical cyclone intensity and track forecasting, combining multiple ML techniques and utilizing diverse data sources. Our multimodal framework, called Hurricast, efficiently combines spatial-temporal data with statistical data by extracting features with deep-learning encoder-decoder architectures and predicting with gradient-boosted trees. We evaluate our models in the North Atlantic and Eastern Pacific basins on 2016-2019 for 24-hour lead time track and intensity forecasts and show they achieve comparable mean absolute error and skill to current operational forecast models while computing in seconds. Furthermore, the inclusion of Hurricast into an operational forecast consensus model could improve over the National Hurricane Center's official forecast, thus highlighting the complementary properties with existing approaches. In summary, our work demonstrates that utilizing machine learning techniques to combine different data sources can lead to new opportunities in tropical cyclone forecasting.
Robustness Test for AI Forecasting of Hurricane Florence Using FourCastNetv2 and Random Perturbations of the Initial Condition
Department of Computer Science, San Diego State University, San Diego, California, United States（计算机科学系，圣地亚哥州立大学，圣地亚哥，加利福尼亚州，美国）；Department of Mathematics, San Diego State University, San Diego, California, United States（数学系，圣地亚哥州立大学，圣地亚哥，加利福尼亚州，美国）
Demand forecasting applications have immensely benefited from the state-of-the-art Deep Learning methods used for time series forecasting. Traditional uni-modal models are predominantly seasonality driven which attempt to model the demand as a function of historic sales along with information on holidays and promotional events. However, accurate and robust sales forecasting calls for accommodating multiple other factors, such as natural calamities, pandemics, elections, etc., impacting the demand for products and product categories in general. We propose a multi-modal sales forecasting network…
Carnegie Mellon University（卡内基梅隆大学）；Mississippi State University（密西西比州立大学）
Tropical cyclone (TC) intensity forecasts are ultimately issued by human forecasters. The human in-the-loop pipeline requires that any forecasting guidance must be easily digestible by TC experts if it is to be adopted at operational centers like the National Hurricane Center. Our proposed framework leverages deep learning to provide forecasters with something neither end-to-end prediction models nor traditional intensity guidance does: a powerful tool for monitoring high-dimensional time series of key physically relevant predictors and the means to understand how the predictors relate to one …
Multimodal Physical Activity Forecasting in Free-Living Clinical Settings: Hunting Opportunities for Just-in-Time Interventions
College of Health Solutions, Arizona State University（亚利桑那州立大学健康解决方案学院）；School of Computing and Augmented Intelligence, Arizona State University（亚利桑那州立大学计算与增强智能学院）；Edson College of Nursing and Health Innovation, Arizona State University（亚利桑那州立大学埃德森护理与健康创新学院）
PA-RNet: Perturbation-Aware Residual Network for Robust Multimodal Time Series Forecasting
Institute of Computing Technology, Guangzhou University（广州大学计算机技术学院）；School of Computer Science and Technology, Dalian University of Technology（大连理工大学计算机科学与技术学院）
University of New South Wales（新南威尔士大学）；University of Sydney（悉尼大学）；The Hong Kong Polytechnic University（香港理工大学）
Considering the multimodal nature of transport systems and potential cross-modal correlations, there is a growing trend of enhancing demand forecasting accuracy by learning from multimodal data. These multimodal forecasting models can improve accuracy but be less practical when different parts of multimodal datasets are owned by different institutions who cannot directly share data among them. While various institutions may can not share their data with each other directly, they may share forecasting models trained by their data, where such models cannot be used to identify the exact informati…
NOAH: Learning the Full Patient Journey. A Longitudinal Multimodal Time-Aware Model for Representation and Forecasting
TUM University Hospital and Technical University of Munich (TUM)（慕尼黑工业大学医院与慕尼黑工业大学）；Institute for Diagnostic and Interventional Radiology, TUM University Hospital（慕尼黑工业大学医院诊断与介入放射学研究所）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；Institute for Diagnostic and Interventional Neuroradiology, TUM University Hospital（慕尼黑工业大学医院诊断与介入神经放射学研究所）；University Medical Center Hamburg-Eppendorf（汉堡埃彭多夫大学医学中心）
SCENARIODIFF: A Scenario-level Guidance Framework for Multimodal Time Series Forecasting--Extended Version
VinUniversity；FPT Software AI Center, FPT Corporation（FPT软件AI中心，FPT集团）；VNU University of Engineering and Technology（VNU工程技术大学）；Aalborg University（奥尔堡大学）
Fidel-TS: A High-Fidelity Multimodal Benchmark for Time Series Forecasting
Department of Computer Science and Engineering（计算机科学与工程系）；The Chinese University of Hong Kong（香港中文大学）；School of Software（软件学院）；Tsinghua University（清华大学）；ZJU-UIUC Institute（浙大-UIUC研究院）；Zhejiang University（浙江大学）；School of Data Science（数据科学学院）；The Chinese University of Hong Kong, Shenzhen（深圳香港中文大学）
National Institute of Standards & Technology（美国国家标准与技术研究院）
Fast and accurate prediction of hurricane evolution from genesis onwards is needed to reduce loss of life and enhance community resilience. In this work, a novel model development methodology for predicting storm trajectory is proposed based on two classes of Recurrent Neural Networks (RNNs). The RNN models are trained on input features available in or derived from the HURDAT2 North Atlantic hurricane database maintained by the National Hurricane Center (NHC). The models use probabilities of storms passing through any location, computed from historical data. A detailed analysis of model foreca…
FuXi-Air: Urban Air Quality Forecasting Based on Emission-Meteorology-Pollutant multimodal Machine Learning
Shanghai Key Laboratory of Atmospheric Particle Pollution and Prevention（上海大气颗粒污染与防治重点实验室）；National Observations and Research Station for Wetland Ecosystems of the Yangtze Estuary（长江入海口湿地生态系统观测与研究站）；Department of Environmental Science and Engineering（环境科学与工程学院）；Shanghai Academy of AI for Science (SAIS)（上海人工智能科学研究院）；MOE Laboratory for National Development and Intelligent Governance（教育部国家发展与智能治理实验室）；Shanghai Institute for Energy and Carbon Neutrality Strategy（上海能源与碳中和战略研究院）；IRDR ICoE on Risk Interconnectivity and Governance on Weather/Climate Extremes Impact and Public Health（国际减灾与发展研究院风险互联与治理国际联合实验室）；Shanghai Institute of Eco-Chongming (SIEC)（上海生态崇明研究院）；Shanghai Environmental Monitoring Center (SEMC)（上海市环境监测中心）
In today's increasingly international economy, return and volatility spillover effects across international equity markets are major macroeconomic drivers of stock dynamics. Thus, information regarding foreign markets is one of the most important factors in forecasting domestic stock prices. However, the cross-correlation between domestic and foreign markets is highly complex. Hence, it is extremely difficult to explicitly express this cross-correlation with a dynamical equation. In this study, we develop stock return prediction models that can jointly consider international markets, using mul…
MM-FinEval: A Multi-Task Multimodal Benchmark for Real-World Financial Forecasting
Northwestern University（西北大学）；New Jersey Institute of Technology（新泽西理工学院）；Georgia Institute of Technology（佐治亚理工学院）；Pace University（佩斯大学）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））
Technical University of Munich（慕尼黑工业大学）；Stanford University（斯坦福大学）；NVIDIA Research（英伟达研究院）
Autonomous systems and humans are increasingly sharing the same space. Robots work side by side or even hand in hand with humans to balance each other's limitations. Such cooperative interactions are ever more sophisticated. Thus, the ability to reason not just about a human's center of gravity position, but also its granular motion is an important prerequisite for human-robot interaction. Though, many algorithms ignore the multimodal nature of humans or neglect uncertainty in their motion forecasts. We present Motron, a multimodal, probabilistic, graph-structured model, that captures human's …
UniDiff: A Unified Diffusion Framework for Multimodal Time Series Forecasting
School of Artificial Intelligence, OPtics and ElectroNics (iOPEN), Northwestern Polytechnical University, Xi’an 710072, China（人工智能学院、光学与电子学（iOPEN）、西北工业大学，西安710072，中国）；Institute of Artificial Intelligence (TeleAI), China Telecom, China（人工智能研究所（TeleAI）、中国电信，中国）
University of Colorado（科罗拉多大学）；Univ. Grenoble Alpes（格勒诺布尔阿尔卑斯大学）；Univ. Savoie Mont Blanc（萨瓦大学）；CNRS（法国国家科学研究中心）；IRD（法国发展研究院）；IFSTTAR（法国交通发展规划与网络研究院）；ISTerre（地球科学研究所）；Linear Accelerator Laboratory（线性加速器实验室）；Université Paris-Sud（巴黎第十一大学）；Inria Saclay–Ile-de-France（法国国家信息与自动化研究所萨克莱-法兰西岛分院）；LRI（信息学实验室）；University of Colorado Boulder（科罗拉多大学博尔德分校）；Cooperative Institute for Research in Environmental Sciences（环境科学合作研究所）；NOAA/OAR/ESRL/Global Systems Division（美国国家海洋和大气管理局海洋与大气研究办公室地球系统研究实验室全球系统部）
The forecast of tropical cyclone trajectories is crucial for the protection of people and property. Although forecast dynamical models can provide high-precision short-term forecasts, they are computationally demanding, and current statistical forecasting models have much room for improvement given that the database of past hurricanes is constantly growing. Machine learning methods, that can capture non-linearities and complex relations, have only been scarcely tested for this application. We propose a neural network model fusing past trajectory data and reanalysis atmospheric images (wind and…
Transformer self-attention encoder-decoder with multimodal deep learning for response time series forecasting and digital twin support in wind structural health monitoring
Department of Mechanical Engineering, University of Bath（巴斯大学机械工程系）；Department of Civil Engineering, Zhejiang University（浙江大学土木工程系）
A multimodal Transformer for InSAR-based ground deformation forecasting with cross-site generalization across Europe
The ADAPT SFI Research Centre, Dublin, Ireland（ADAPT SFI研究机构，都柏林，爱尔兰）；School of Computer Science, University College Dublin, Belfield, Dublin, Ireland（计算机科学学院，都柏林大学学院，贝尔菲德，都柏林，爱尔兰）
TyphoFormer: Language-Augmented Transformer for Accurate Typhoon Track Forecasting
Florida State University（佛罗里达州立大学）；FAMU-FSU College of Engineering（FAMU-FSU 工程学院）；University of Southern California（南加州大学）；University of Maryland（马里兰大学）
Microsoft（微软）；Department of Atmospheric Sciences, University of Washington（华盛顿大学大气科学系）；Microsoft Research（微软研究院）
We present an ensemble prediction system using a Deep Learning Weather Prediction (DLWP) model that recursively predicts key atmospheric variables with six-hour time resolution. This model uses convolutional neural networks (CNNs) on a cubed sphere grid to produce global forecasts. The approach is computationally efficient, requiring just three minutes on a single GPU to produce a 320-member set of six-week forecasts at 1.4° resolution. Ensemble spread is primarily produced by randomizing the CNN training process to create a set of 32 DLWP models with slightly different learned weights. Althou…
