变分数据同化通过最小化使数值模型与观测数据相拟合的代价函数来估计动力系统状态。尽管四维变分同化(4D-Var)应用广泛,但它在复杂非线性系统中存在计算成本高的问题,且依赖不完善的状态-观测映射。深度学习(DL)可提供表达能力更强的近似器,但由于其非线性特性以及同化结果缺乏理论保证,将深度学习模型集成到4D-Var中颇具挑战。本文提出Tensor-Var,这是一种将核条件均值嵌入(CME,kernel conditional mean embedding)与4D-Var相结合的新型框架,可对非线性动力学进行线性化,在学习到的特征空间中实现凸优化。此外,我们的方法为以线性方式求解4D-Var提供了新视角,为原始空间与特征空间之间的同化结果一致性提供了理论保证。为处理大规模问题,我们提出了一种在Tensor-Var框架内使用神经网络学习深度特征的方法。在混沌系统以及结合实时观测的全球天气预测任务上的实验表明,Tensor-Var在精度上优于传统的以及深度学习混合的4D-Var基线方法,同时实现了10到20倍的速度提升。
英文摘要:
Variational data assimilation estimates the dynamical system states by minimizing a cost function that fits the numerical models with the observational data. Although four-dimensional variational assimilation (4D-Var) is widely used, it faces high computational costs in complex nonlinear systems and depends on imperfect state-observation mappings. Deep learning (DL) offers more expressive approximators, while integrating DL models into 4D-Var is challenging due to their nonlinearities and lack of theoretical guarantees in assimilation results. In this paper, we propose Tensor-Var, a novel framework that integrates kernel conditional mean embedding (CME) with 4D-Var to linearize nonlinear dynamics, achieving convex optimization in a learned feature space. Moreover, our method provides a new perspective for solving 4D-Var in a linear way, offering theoretical guarantees of consistent assimilation results between the original and feature spaces. To handle large-scale problems, we propose a method to learn deep features using neural networks within the Tensor-Var framework. Experiments on chaotic systems and global weather prediction with real-time observations show that Tensor-Var outperforms conventional and DL hybrid 4D-Var baselines in accuracy while achieving a 10- to 20-fold speed improvement.
Surrogate-based variational data assimilation for tidal modelling
EDF R&D(法国电力集团研发中心);European Center for Research and Advanced Training in Scientific Computation (CERFACS)(欧洲科学计算高级研究与培训中心);French National Research Center (CNRS)(法国国家科学研究中心);Université de Toulouse(图卢兹大学);Saint-Venant Laboratory for Hydraulics (LHSV)(圣维南水力学实验室)
Multi-gauge Hydrological Variational Data Assimilation: Regionalization Learning with Spatial Gradients using Multilayer Perceptron and Bayesian-Guided Multivariate Regression
INRAE(法国国家农业食品与环境研究院);Aix-Marseille Université(艾克斯-马赛大学);Institut de Mécanique des Fluides de Toulouse (IMFT)(图卢兹流体力学研究所);Université de Toulouse(图卢兹大学);CNRS(法国国家科学研究中心)
Meta-Learning Fourier Neural Operators for Hessian Inversion and Enhanced Variational Data Assimilation
School of Computational Science and Engineering, McMaster University(计算科学与工程学院,麦斯特大学);Mathematics and Statistics, McMaster University(数学与统计学,麦斯特大学);Department of Chemistry and Chemical Biology, McMaster University(化学与生物化学系,麦斯特大学)
本文提出元学习 FNO 框架近似跨 DA 问题的逆 Hessian,以初始化 CG;线性平流实验显示 FNO-CG 显著降低误差与迭代数,尤其适用于病态场景。
PnP-DA: Towards Principled Plug-and-Play Integration of Variational Data Assimilation and Generative Models
Department of Earth and Environmental Engineering, Columbia University, New York, USA(哥伦比亚大学地球与环境工程系);NSF Center for Learning the Earth with Artificial Intelligence and Physics (LEAP), New York, USA(国家科学基金会学习地球与人工智能和物理中心(LEAP));Pasteur Labs, New York, USA(帕斯特实验室);Courant Institute for Mathematical Science, New York University, New York, USA(纽约大学Courant数学科学研究所)
Enhancing the Parameterization of Reservoir Properties for Data Assimilation Using Deep VAE-GAN
Departamento de Engenharia de Minas e de Petróleo, Escola Politécnica, Universidade de São Paulo(圣保罗大学采矿与石油工程系,理工学院);Department of Earth Science and Engineering, Imperial College London(伦敦帝国理工学院地球科学与工程系)
Efficient deep data assimilation with sparse observations and time-varying sensors
Imperial College London(帝国理工学院);Leverhulme Centre for Wildfires, Environment, and Society(勒沃霍姆野火、环境与社会研究中心);Hong Kong University of Science and Technology(香港科技大学)
AI enhanced data assimilation and uncertainty quantification applied to Geological Carbon Storage
Faculty of Civil Engineering;Geosciences, TU Delft, Stevinweg 1, 2628 CN Delft, Netherlands;Centrum Wiskunde \& Informatica, Science Park 123, 1098 XG Amsterdam, Netherlands;Mathematical Institute, Utrecht University, Utrecht, Netherlands;Department of Earth Science;Engineering, Imperial College London, London, United Kingdom;Department of Energy Resources Engineering, Stanford University, CA, USA
Marine Physical Laboratory (Scripps Institution of Oceanography), University of California, San Diego(加州大学圣迭戈分校斯克里普斯海洋研究所海洋物理实验室);Department of Physics, University of California, San Diego(加州大学圣迭戈分校物理系)
U-Net Kalman Filter (UNetKF): An Example of Machine Learning-assisted Ensemble Data Assimilation
University Cooperation of Atmospheric Research(大学大气研究合作组织);National Oceanic and Atmospheric Administration/Geophysical Fluid Dynamics Laboratory(美国国家海洋和大气管理局/地球物理流体动力学实验室);Princeton University(普林斯顿大学)
Mitigating loss of variance in ensemble data assimilation: machine learning-based and distance-free localization
Novel Reservoir Modelling and Simulation Group, Imperial College London(帝国理工学院伦敦分校新型油藏建模与模拟小组);Faculty of Civil Engineering and Geosciences, TU Delft(代尔夫特理工大学土木工程与地质科学学院)