Prithvi WxC：面向天气与气候的基础模型
Prithvi WxC: Foundation Model for Weather and Climate
AI总结：
研究填补了天气气候领域缺乏多用途基础模型的空白，提出基于编码器-解码器架构、拥有23亿参数的开源基础模型Prithvi WxC，结合掩码重建与预报混合训练，在多个下游任务上表现优异。
摘要：
受到AI模拟器可以匹敌在HPC系统上运行的传统数值天气预报模型性能这一认识的推动，目前有越来越多的大型AI模型用于解决预报、降尺度或临近预报等用例。尽管AI文献中的并行发展专注于基础模型——即可以有效微调以解决多种不同用例的模型——但天气与气候方面的发展很大程度上专注于单一用例，特别强调中短期预报。我们通过引入Prithvi WxC来填补这一空白，这是一个拥有23亿参数的基础模型，使用Modern-Era Retrospective Analysis for Research and Applications, Version 2（MERRA-2）中的160个变量开发。Prithvi WxC采用基于编码器-解码器的架构，结合了各种近期transformer模型的概念，以有效捕获输入数据中的区域和全局依赖关系。该模型被设计为可容纳大量token，以在不同拓扑结构中对精细分辨率的天气现象进行建模。此外，它使用混合目标进行训练，将掩码重建与预报范式相结合。我们在一组具有挑战性的下游任务上测试了该模型，即：自回归展开预报、降尺度、重力波通量参数化和极端事件估计。这个拥有23亿参数的预训练模型及其相关的微调工作流已作为开源贡献通过Hugging Face公开发布。
英文摘要：
Triggered by the realization that AI emulators can rival the performance of traditional numerical weather prediction models running on HPC systems, there is now an increasing number of large AI models that address use cases such as forecasting, downscaling, or nowcasting. While the parallel developments in the AI literature focus on foundation models -- models that can be effectively tuned to address multiple, different use cases -- the developments on the weather and climate side largely focus on single-use cases with particular emphasis on mid-range forecasting. We close this gap by introducing Prithvi WxC, a 2.3 billion parameter foundation model developed using 160 variables from the Modern-Era Retrospective Analysis for Research and Applications, Version 2 (MERRA-2). Prithvi WxC employs an encoder-decoder-based architecture, incorporating concepts from various recent transformer models to effectively capture both regional and global dependencies in the input data. The model has been designed to accommodate large token counts to model weather phenomena in different topologies at fine resolutions. Furthermore, it is trained with a mixed objective that combines the paradigms of masked reconstruction with forecasting. We test the model on a set of challenging downstream tasks namely: Autoregressive rollout forecasting, Downscaling, Gravity wave flux parameterization, and Extreme events estimation. The pretrained model with 2.3 billion parameters, along with the associated fine-tuning workflows, has been publicly released as an open-source contribution via Hugging Face.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
