In this paper, we introduce a new, local formulation of the ensemble Kalman Filter approach for atmospheric data assimilation. Our scheme is based on the hypothesis that, when the Earth's surface is divided up into local regions of moderate size, vectors of the forecast uncertainties in such regions tend to lie in a subspace of much lower dimension than that of the full atmospheric state vector of such a region. Ensemble Kalman Filters, in general, assume that the analysis resulting from the data assimilation lies in the same subspace as the expected forecast error. Under our hypothesis the dimension of this subspace is low. This implies that operations only on relatively low dimensional matrices are required. Thus, the data analysis is done locally in a manner allowing massively parallel computation to be exploited. The local analyses are then used to construct global states for advancement to the next forecast time. The method, its potential advantages, properties, and implementation requirements are illustrated by numerical experiments on the Lorenz-96 model. It is found that accurate analysis can be achieved at a cost which is very modest compared to that of a full global ensemble Kalman Filter.
U-Net Kalman Filter (UNetKF): An Example of Machine Learning-assisted Ensemble Data Assimilation
University Cooperation of Atmospheric Research(大学大气研究合作组织);National Oceanic and Atmospheric Administration/Geophysical Fluid Dynamics Laboratory(美国国家海洋和大气管理局/地球物理流体动力学实验室);Princeton University(普林斯顿大学)
On temporal scale separation in coupled data assimilation with the ensemble Kalman filter
耦合资料同化(CDA)是数值天气和气候预报领域的一个重要研究方向,在世界范围内得到了广泛的应用。核心问题是尺度分离作为一个障碍,阻碍了跨模型组件的信息传播。我们提供了一个简短的调查CDA,然后集中在CDA使用集合卡尔曼滤波器(EnKF)。我们考虑第一耦合方程的时间尺度差异,并推导出:(i)交叉组件的影响是强的从慢到快的尺度,但(ii)内组件的影响是更强的快尺度。虽然观察慢标度是可取的,并有利于快标度,但后者必须以高频率观察,否则误差将影响慢标度。实验使用的大气-海洋模式,MAOOAM。六个配置被认为是不同的大气-海洋耦合的强度和/或模式的数量。通过研究李雅普诺夫谱、柯尔莫哥洛夫熵和Kaplan-Yorke吸引子维数,给出了该模型的动力学特征。我们还计算了协变的李雅普诺夫向量,并用它们来解释模型不稳定性如何根据耦合强度对不同模型的模式产生作用。实验证实了观测快速尺度的重要性,但也表明,尽管其时间尺度缓慢,但在海洋中进行频繁的观测是有益的。研究了系综大小与不稳定子空间维数之间的关系。结果在很大程度上证实了非耦合系统中已知的结论:N> n 0是EnKF收敛的必要条件。但MAOOAM的李雅普诺夫谱的准简并性,具有许多接近零的指数,可能是某些模型配置观察到的分析误差平滑逐渐减小的原因,即使当N> n 0。
Physically Consistent Global Atmospheric Data Assimilation with Machine Learning in Latent Space
Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Department of Earth and Environmental Engineering, Columbia University(哥伦比亚大学地球与环境工程系);Learning the Earth with Artificial intelligence and Physics (LEAP) Center(人工智能与物理学习地球中心(LEAP));Nanjing University of Information Science and Technology(南京信息工程大学);The Chinese University of Hong Kong(香港中文大学)
Optimal and scalable methods to approximate the solutions of large-scale Bayesian problems: Theory and application to atmospheric inversions and data assimilation
LO-SDA: Latent Optimization for Score-based Atmospheric Data Assimilation
Fudan University(复旦大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Columbia University(哥伦比亚大学);The Chinese University of Hong Kong(香港中文大学);Shanghai Jiaotong University(上海交通大学)
Online machine-learning forecast uncertainty estimation for sequential data assimilation
Servicio Meteorológico Nacional(国家气象局);Universidad de Buenos Aires(布宜诺斯艾利斯大学);Instituto Franco-Argentino para el Estudio del Clima y sus Impactos (IRL 3351 IFAECI)(法阿气候及其影响研究所(IRL 3351 IFAECI));Universidad Nacional del Nordeste(东北国立大学);Centro de Investigaciones del Mar y la Atmósfera(海洋与大气研究中心);IMT Atlantique(大西洋国立高等工业技术学院);Inria(法国国家信息与自动化研究所)
Align-DA: Align Score-based Atmospheric Data Assimilation with Multiple Preferences
Fudan University(复旦大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);The Chinese University of Hong Kong(香港中文大学);Shanghai Jiaotong University(上海交通大学);Columbia University(哥伦比亚大学)
Nansen Environmental and Remote Sensing Center(南森环境与遥感中心);Sorbonne Université(索邦大学);Universitetet i Bergen(卑尔根大学);Bjerknes Center for Climate Research(比耶克内斯气候研究中心)
Accurate and Efficient Hybrid-Ensemble Atmospheric Data Assimilation in Latent Space with Uncertainty Quantification
Department of Earth and Environmental Engineering, School of Engineering and Applied Sciences, Climate School, Columbia University(地球与环境工程系,工程与应用科学学院,气候学院,哥伦比亚大学);Learning the Earth with Artificial Intelligence and Physics (LEAP) Center, Columbia University(人工智能与物理学习地球中心,哥伦比亚大学);Shanghai Artificial Intelligence Laboratory, Shanghai, China(上海人工智能实验室,上海,中国);The Chinese University of Hong Kong, Hong Kong, China(香港中文大学,香港,中国);Department of Computer Science and Technology, Tsinghua University, Beijing, China(计算机科学与技术系,清华大学,北京,中国)
Building a Fuel Moisture Model for the Coupled Fire-Atmosphere Model WRF-SFIRE from Data: From Kalman Filters to Recurrent Neural Networks
University of Colorado Denver(科罗拉多大学丹佛分校);San José State University(圣何塞州立大学);Colorado State University(科罗拉多州立大学);University of Utah(犹他大学);University of Southern California(南加州大学)
Center for Environmental Remote Sensing, Chiba University(千叶大学环境遥感中心);Meteorological Research Institute, Japan Meteorological Agency(日本气象厅气象研究所);Institute for Advanced Academic Research, Chiba University(千叶大学高级学术研究学院);Research Institute of Disaster Medicine, Chiba University(千叶大学灾害医学研究所)
Combining data assimilation and machine learning to emulate a dynamical model from sparse and noisy observations: a case study with the Lorenz 96 model
Sorbonne University(索邦大学);University of Reading(雷丁大学);University of Utrecht(乌得勒支大学);École des Ponts ParisTech(巴黎高科路桥学院);EDF R&D(法国电力集团研发部);Université Paris-Est(巴黎东大学)