Coupled data assimilation (CDA) distinctively appears as a main concern in numerical weather and climate prediction with major efforts put forward worldwide. The core issue is the scale separation acting as a barrier that hampers the propagation of the information across model components. We provide a brief survey of CDA, and then focus on CDA using the ensemble Kalman filter (EnKF). We consider first coupled equations with temporal scale difference and deduce that: (i) cross components effects are strong from the slow to the fast scale, but, (ii) intra-component effects are much stronger in the fast scale. While observing the slow scale is desirable and benefits the fast, the latter must be observed with high frequency otherwise the error will affect the slow scale. Experiments are performed using the atmosphere-ocean model, MAOOAM. Six configurations are considered, differing for the strength of the atmosphere-ocean coupling and/or the number of model modes. A comprehensive dynamical characterisation of the model configurations is provided by examining the Lyapunov spectrum, Kolmogorov entropy and Kaplan-Yorke attractor dimension. We also compute the covariant Lyapunov vectors and use them to explain how model instabilities act on different model's modes according to the coupling strength. The experiments confirm the importance of observing the fast scale, but show also that, despite its slow temporal scale, frequent observations in the ocean are beneficial. The relation between the ensemble size and the unstable subspace dimension has been studied. Results largely ratify what known for uncoupled system: the condition N>n0 is necessary for the EnKF to converge. But the quasi-degeneracy of the Lyapunov spectrum of MAOOAM, with many near-zero exponents, is potentially the cause of the smooth gradual reduction of the analysis error observed for some model configurations, even when N>n0.
Data assimilation is an iterative approach to the problem of estimating the state of a dynamical system using both current and past observations of the system together with a model for the system's time evolution. Rather than solving the problem from scratch each time new observations become available, one uses the model to ``forecast'' the current state, using a prior state estimate (which incorporates information from past data) as the initial condition, then uses current data to correct the prior forecast to a current state estimate. This Bayesian approach is most effective when the uncerta…
We explore the potential of three-dimensional data assimilation for assimilating sparsely-distributed 2-metre temperature observations across the coupled atmosphere-land interface into the soil moisture. Using idealised twin experiments with the limited-area modelling platform TerrSysMP and synthetic observations, we avoid model biases and directly control errors in the initial conditions and observations. These experiments allow us to test hourly data assimilation with a localised ensemble Kalman filter, as often used for mesoscale data assimilation. We find here an error reduction of such an…
In this paper, we introduce a new, local formulation of the ensemble Kalman Filter approach for atmospheric data assimilation. Our scheme is based on the hypothesis that, when the Earth's surface is divided up into local regions of moderate size, vectors of the forecast uncertainties in such regions tend to lie in a subspace of much lower dimension than that of the full atmospheric state vector of such a region. Ensemble Kalman Filters, in general, assume that the analysis resulting from the data assimilation lies in the same subspace as the expected forecast error. Under our hypothesis the di…
We show that modifying a Bayesian data assimilation scheme by incorporating kinematically-consistent displacement corrections produces a scheme that is demonstrably better at estimating partially observed state vectors in a setting where feature information important. While the displacement transformation is not tied to any particular assimilation scheme, here we implement it within an ensemble Kalman Filter and demonstrate its effectiveness in tracking stochastically perturbed vortices.
The feasibility of global ocean state estimation by sequential data assimilation is demonstrated. The model componenet of the assimilator is the GROB version of the MPIMET ocean circulation model HOPE. Assimilation uses the Fokker-Planck representation of the Kalman Filter. This approach determines the temporal evolution of error statistics by integration of the Fokker-Planck Equation. Phase space advection and diffusion are obtained from histogram techniques considering the model as a black box. For efficiency, the estimate combines nudging and Kalman Filtering. The ocean state is estimated f…
An Ensemble Kalman Filter (EnKF, the predictor) is used make a large change in the state, followed by a Particle Filer (PF, the corrector) which assigns importance weights to describe non-Gaussian distribution. The weights are obtained by nonparametric density estimation. It is demonstrated on several numerical examples that the new predictor-corrector filter combines the advantages of the EnKF and the PF and that it is suitable for high dimensional states which are discretizations of solutions of partial differential equations.
Ensemble data assimilation methods such as the Ensemble Kalman Filter (EnKF) are a key component of probabilistic weather forecasting. They represent the uncertainty in the initial conditions by an ensemble which incorporates information coming from the physical model with the latest observations. High-resolution numerical weather prediction models ran at operational centers are able to resolve non-linear and non-Gaussian physical phenomena such as convection. There is therefore a growing need to develop ensemble assimilation algorithms able to deal with non-Gaussianity while staying computati…
Data assimilation (DA) aims to optimally combine model forecasts and observations that are both partial and noisy. Multi-model DA generalizes the variational or Bayesian formulation of the Kalman filter, and we prove that it is also the minimum variance linear unbiased estimator. Here, we formulate and implement a multi-model ensemble Kalman filter (MM-EnKF) based on this framework. The MM-EnKF can combine multiple model ensembles for both DA and forecasting in a flow-dependent manner; it uses adaptive model error estimation to provide matrix-valued weights for the separate models and the obse…
U-Net Kalman Filter (UNetKF): An Example of Machine Learning-assisted Ensemble Data Assimilation
University Cooperation of Atmospheric Research（大学大气研究合作组织）；National Oceanic and Atmospheric Administration/Geophysical Fluid Dynamics Laboratory（美国国家海洋和大气管理局/地球物理流体动力学实验室）；Princeton University（普林斯顿大学）
We present a new type of the EnKF for data assimilation in spatial models that uses diagonal approximation of the state covariance in the wavelet space to achieve adaptive localization. The efficiency of the new method is demonstrated on an example.
Brazilian National Institute of Space Research（巴西国家空间研究院）
This paper presents an approach for employing artificial neural networks (NN) to emulate an ensemble Kalman filter (EnKF) as a method of data assimilation. The assimilation methods are tested in the Simplified Parameterizations PrimitivE-Equation Dynamics (SPEEDY) model, an atmospheric general circulation model (AGCM), using synthetic observational data simulating localization of balloon soundings. For the data assimilation scheme, the supervised NN, the multilayer perceptrons (MLP-NN), is applied. The MLP-NN are able to emulate the analysis from the local ensemble transform Kalman filter (LET…
We consider the problem of an ensemble Kalman filter when only partial observations are available. In particular we consider the situation where the observational space consists of variables which are directly observable with known observational error, and of variables of which only their climatic variance and mean are given. To limit the variance of the latter poorly resolved variables we derive a variance limiting Kalman filter (VLKF) in a variational setting. We analyze the variance limiting Kalman filter for a simple linear toy model and determine its range of optimal performance. We explo…
The ensemble Kalman filter (EnKF) is a recursive filter suitable for problems with a large number of variables, such as discretizations of partial differential equations in geophysical models. The EnKF originated as a version of the Kalman filter for large problems (essentially, the covariance matrix is replaced by the sample covariance), and it is now an important data assimilation component of ensemble forecasting. EnKF is related to the particle filter (in this context, a particle is the same thing as an ensemble member) but the EnKF makes the assumption that all probability distributions i…
Ensemble methods such as the Ensemble Kalman Filter (EnKF) are widely used for data assimilation in large-scale geophysical applications, as for example in numerical weather prediction (NWP). There is a growing interest for physical models with higher and higher resolution, which brings new challenges for data assimilation techniques because of the presence of non-linear and non-Gaussian features that are not adequately treated by the EnKF. We propose two new localized algorithms based on the Ensemble Kalman Particle Filter (EnKPF), a hybrid method combining the EnKF and the Particle Filter (P…
The ensemble Kalman filter (EnKF) is an efficient algorithm for many data assimilation problems. In certain circumstances, however, divergence of the EnKF might be spotted. In previous studies, the authors proposed an observation-space-based strategy, called residual nudging, to improve the stability of the EnKF when dealing with linear observation operators. The main idea behind residual nudging is to monitor and, if necessary, adjust the distances (misfits) between the real observations and the simulated ones of the state estimates, in the hope that by doing so one may be able to obtain bett…
The understanding of nonlinear, high dimensional flows, e.g, atmospheric and ocean flows, is critical to address the impacts of global climate change. Data Assimilation techniques combine physical models and observational data, often in a Bayesian framework, to predict the future state of the model and the uncertainty in this prediction. Inherent in these systems are noise (Gaussian and non-Gaussian), nonlinearity, and high dimensionality that pose challenges to making accurate predictions. To address these issues we investigate the use of both model and data dimension reduction based on techn…
Online machine-learning forecast uncertainty estimation for sequential data assimilation
Servicio Meteorológico Nacional（国家气象局）；Universidad de Buenos Aires（布宜诺斯艾利斯大学）；Instituto Franco-Argentino para el Estudio del Clima y sus Impactos (IRL 3351 IFAECI)（法阿气候及其影响研究所（IRL 3351 IFAECI））；Universidad Nacional del Nordeste（东北国立大学）；Centro de Investigaciones del Mar y la Atmósfera（海洋与大气研究中心）；IMT Atlantique（大西洋国立高等工业技术学院）；Inria（法国国家信息与自动化研究所）
University of Colorado Denver（科罗拉多大学丹佛分校）；San José State University（圣何塞州立大学）；Colorado State University（科罗拉多州立大学）；University of Utah（犹他大学）；University of Southern California（南加州大学）
The current fuel moisture content (FMC) subsystems in WRF-SFIRE and its workflow system WRFx use a time-lag differential equation model with assimilation of data from FMC sensors on Remote Automated Weather Stations (RAWS) by the extended augmented Kalman filter. But the quality of the result is constrained by the limitations of the model and of the Kalman filter. We observe that the data flow in a system consisting of a model and the Kalman filter can be interpreted to be the same as the data flow in a recurrent neural network (RNN). Thus, instead of building more sophisticated models and dat…
Convergence of the ensemble Kalman filter in the limit for large ensembles to the Kalman filter is proved. In each step of the filter, convergence of the ensemble sample covariance follows from a weak law of large numbers for exchangeable random variables, the continuous mapping theorem gives convergence in probability of the ensemble members, and $L^p$ bounds on the ensemble then give $L^p$ convergence.
