The rapid rise of deep learning (DL) in numerical weather prediction (NWP) has led to a proliferation of models which forecast atmospheric variables with comparable or superior skill than traditional physics-based NWP. However, among these leading DL models, there is a wide variance in both the training settings and architecture used. Further, the lack of thorough ablation studies makes it hard to discern which components are most critical to success. In this work, we show that it is possible to attain high forecast skill even with relatively off-the-shelf architectures, simple training procedures, and moderate compute budgets. Specifically, we train a minimally modified SwinV2 transformer on ERA5 data, and find that it attains superior forecast skill when compared against IFS. We present some ablations on key aspects of the training pipeline, exploring different loss functions, model sizes and depths, and multi-step fine-tuning to investigate their effect. We also examine the model performance with metrics beyond the typical ACC and RMSE, and investigate how the performance scales with model size.
How many stations are sufficient? Exploring the effect of urban weather station density reduction on imputation accuracy of air temperature and humidity
Biometry and Environmental System Analysis, University of Freiburg, Germany(弗赖堡大学生物计量与环境系统分析系);Chair of Environmental Meteorology, University of Freiburg, Germany(弗赖堡大学环境气象学教授职位)
Exploring the impact of weather on Metro demand forecasting using machine learning method
Qiushi Honors College, Tianjin University(天津大学求是学部);School of Traffic and Transportation, Beijing Jiaotong University(北京交通大学交通运输学院);Maynooth International Engineering College, Fuzhou University(福州大学梅努斯国际工程学院);Harbin Institute of Technology(哈尔滨工业大学);Beijing Forestry University(北京林业大学)
PINN-Cast: Exploring the Role of Continuous-Depth NODE in Transformers and Physics Informed Loss as Soft Physical Constraints in Short-term Weather Forecasting
Exploring Quantum Machine Learning for Weather Forecasting
QuIIN—Quantum Industrial Innovation, EMBRAPII CIMATEC Competence Center in Quantum Technologies, SENAI CIMATEC(量子工业创新机构、EMBRAPII CIMATEC量子技术能力中心、SENAI CIMATEC);Quantum Information Group, Federal University of Western Bahia - Campus Reitor Edgard Santos(量子信息组、巴伊亚西部联邦大学)
CommonAccent: Exploring Large Acoustic Pretrained Models for Accent Classification Based on Common Voice
Idiap Research Institute(Idiap研究所);Ecole Polytechnique Federale de Lausanne(洛桑联邦理工学院);Texas A&M University(得克萨斯农工大学);Vilnius Gediminas Technical University(维尔纽斯盖迪米纳斯技术大学);Université Laval(拉瓦尔大学);Concordia University(康考迪亚大学);Mila-Québec AI Institute(米拉-魁北克人工智能研究所)
Weather Prediction Using CNN-LSTM for Time Series Analysis: A Case Study on Delhi Temperature Data
Glasgow College, University of Electronic Science and Technology of China(电子科技大学格拉斯哥学院);Viterbi School of Engineering, University of Southern California(南加州大学维特比工程学院)
Towards Efficient Pre-training: Exploring FP4 Precision in Large Language Models
School of Information Science and Technology, University of Science and Technology of China(中国科学技术大学信息科学技术学院);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室)