A recent line of work in the machine learning community addresses the problem of predicting high-dimensional spatiotemporal phenomena by leveraging specific tools from the differential equations theory. Following this direction, we propose in this article a novel and general paradigm for this task based on a resolution method for partial differential equations: the separation of variables. This inspiration allows us to introduce a dynamical interpretation of spatiotemporal disentanglement. It induces a principled model based on learning disentangled spatial and temporal representations of a phenomenon to accurately predict future observations. We experimentally demonstrate the performance and broad applicability of our method against prior state-of-the-art models on physical and synthetic video datasets.
In this paper, we consider the problem of learning prediction models for spatiotemporal physical processes driven by unknown partial differential equations (PDEs). We propose a deep learning framework that learns the underlying dynamics and predicts its evolution using sparsely distributed data sites. Deep learning has shown promising results in modeling physical dynamics in recent years. However, most of the existing deep learning methods for modeling physical dynamics either focus on solving known PDEs or require data in a dense grid when the governing PDEs are unknown. In contrast, our meth…
Huazhong University of Science and Technology（华中科技大学）；KTH Royal Institute of Technology（瑞典皇家理工学院）；Georgia Institute of Technology（佐治亚理工学院）；Wuhan University（武汉大学）；University of Cambridge（剑桥大学）；University of Luxembourg（卢森堡大学）；Weill Cornell Medical College（威尔康奈尔医学院）
The study presents a general framework for discovering underlying Partial Differential Equations (PDEs) using measured spatiotemporal data. The method, called Sparse Spatiotemporal System Discovery ($\text{S}^3\text{d}$), decides which physical terms are necessary and which can be removed (because they are physically negligible in the sense that they do not affect the dynamics too much) from a pool of candidate functions. The method is built on the recent development of Sparse Bayesian Learning; which enforces the sparsity in the to-be-identified PDEs, and therefore can balance the model compl…
Modeling the dynamics of real-world physical systems is critical for spatiotemporal prediction tasks, but challenging when data is limited. The scarcity of real-world data and the difficulty in reproducing the data distribution hinder directly applying meta-learning techniques. Although the knowledge of governing partial differential equations (PDE) of data can be helpful for the fast adaptation to few observations, it is mostly infeasible to exactly find the equation for observations in real-world physical systems. In this work, we propose a framework, physics-aware meta-learning with auxilia…
Sorbonne Université（索邦大学）；CNRS（法国国家科学研究中心）；Criteo AI Lab（Criteo人工智能实验室）
Effective data-driven PDE forecasting methods often rely on fixed spatial and / or temporal discretizations. This raises limitations in real-world applications like weather prediction where flexible extrapolation at arbitrary spatiotemporal locations is required. We address this problem by introducing a new data-driven approach, DINo, that models a PDE's flow with continuous-time dynamics of spatially continuous functions. This is achieved by embedding spatial observations independently of their discretization via Implicit Neural Representations in a small latent space temporally driven by a l…
In this work, we present a machine learning approach for reducing the error when numerically solving time-dependent partial differential equations (PDE). We use a fully convolutional LSTM network to exploit the spatiotemporal dynamics of PDEs. The neural network serves to enhance finite-difference and finite-volume methods (FDM/FVM) that are commonly used to solve PDEs, allowing us to maintain guarantees on the order of convergence of our method. We train the network on simulation data, and show that our network can reduce error by a factor of 2 to 3 compared to the baseline algorithms. We dem…
We explore the derivation of distributed parameter system evolution laws (and in particular, partial differential operators and associated partial differential equations, PDEs) from spatiotemporal data. This is, of course, a classical identification problem; our focus here is on the use of manifold learning techniques (and, in particular, variations of Diffusion Maps) in conjunction with neural network learning algorithms that allow us to attempt this task when the dependent variables, and even the independent variables of the PDE are not known a priori and must be themselves derived from the …
Solving partial differential equations (PDEs) is an important research means in the fields of physics, biology, and chemistry. As an approximate alternative to numerical methods, PINN has received extensive attention and played an important role in many fields. However, PINN uses a fully connected network as its model, which has limited fitting ability and limited extrapolation ability in both time and space. In this paper, we propose PhyGNNet for solving partial differential equations on the basics of a graph neural network which consists of encoder, processer, and decoder blocks. In particul…
PhyMPGN: Physics-encoded Message Passing Graph Network for spatiotemporal PDE systems
Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学高瓴人工智能学院）；School of Engineering Science, University of Chinese Academy of Sciences（中国科学院大学工程科学学院）；Huawei Technologies（华为技术有限公司）
Imperial College London（帝国理工学院）；The Alan Turing Institute（艾伦·图灵研究所）；University of Warwick（华威大学）；University of Bath（巴斯大学）
Stochastic partial differential equations (SPDEs) are the mathematical tool of choice for modelling spatiotemporal PDE-dynamics under the influence of randomness. Based on the notion of mild solution of an SPDE, we introduce a novel neural architecture to learn solution operators of PDEs with (possibly stochastic) forcing from partially observed data. The proposed Neural SPDE model provides an extension to two popular classes of physics-inspired architectures. On the one hand, it extends Neural CDEs and variants -- continuous-time analogues of RNNs -- in that it is capable of processing incomi…
Northeastern University（东北大学）；University of Notre Dame（圣母大学）；MIT（麻省理工学院）
Partial differential equations (PDEs) play a fundamental role in modeling and simulating problems across a wide range of disciplines. Recent advances in deep learning have shown the great potential of physics-informed neural networks (PINNs) to solve PDEs as a basis for data-driven modeling and inverse analysis. However, the majority of existing PINN methods, based on fully-connected NNs, pose intrinsic limitations to low-dimensional spatiotemporal parameterizations. Moreover, since the initial/boundary conditions (I/BCs) are softly imposed via penalty, the solution quality heavily relies on h…
Microsoft Autonomous Systems and Robotics Research（微软自主系统与机器人研究部）；Microsoft Research AI4Science（微软研究院AI4Science）
Partial differential equations (PDEs) are central to describing complex physical system simulations. Their expensive solution techniques have led to an increased interest in deep neural network based surrogates. However, the practical utility of training such surrogates is contingent on their ability to model complex multi-scale spatio-temporal phenomena. Various neural network architectures have been proposed to target such phenomena, most notably Fourier Neural Operators (FNOs), which give a natural handle over local & global spatial information via parameterization of different Fourier mode…
P$^2$C$^2$Net: PDE-Preserved Coarse Correction Network for efficient prediction of spatiotemporal dynamics
Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学高瓴人工智能学院）；Northeastern University（东北大学）；University of Notre Dame（圣母大学）；Huawei Technologies（华为技术有限公司）；School of Engineering Science, University of Chinese Academy of Sciences（中国科学院大学工程科学学院）
RBF-MGN:Solving spatiotemporal PDEs with Physics-informed Graph Neural Network
College of Aerospace Science and Engineering, National University of Defense Technology（国防科技大学航天科学与工程学院）；National Innovation Institute of Defense Technology, Chinese Academy of Military Science（中国人民解放军军事科学院国防科技创新研究院）
Multi-resolution partial differential equations preserved learning framework for spatiotemporal dynamics
University of Notre Dame（圣母大学）；Yale University（耶鲁大学）；Renmin University of China（中国人民大学）；Beijing Key Laboratory of Big Data Management and Analysis Methods（大数据管理与分析方法北京市重点实验室）
Mechanistic PDE Networks for Discovery of Governing Equations
Informatics Institute, University of Amsterdam, Amsterdam, The Netherlands（阿姆斯特丹大学信息技术研究所）；Institute of Science and Technology, Klosterneuburg, Austria（科技研究院）
A Spectral Approach for Learning Spatiotemporal Neural Differential Equations
Courant Institute of Mathematical Sciences, New York University, New York, NY 10012, USA（数学科学学院，纽约大学，纽约，NY 10012，美国）；Department of Computational Medicine, UCLA, Los Angeles, CA 90095, USA（计算医学系，洛杉矶大学，洛杉矶，CA 90095，美国）；Nuffield Department of Medicine, Oxford University, Oxford OX2 6HW, UK（医学系，牛津大学，牛津 OX2 6HW，英国）
Monte Carlo Neural PDE Solver for Learning PDEs via Probabilistic Representation
Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学高瓴人工智能学院）；Academy of Mathematics and Systems Science, Chinese Academy of Sciences (CAS)（中国科学院数学与系统科学研究院）；Beijing Institute of Technology（北京理工大学）；Zhongguancun Academy（中关村学院）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）
Stable spectral neural operator for learning stiff PDE systems from limited data
Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学人工智能学院（ Gallagher 学院））；School of Engineering Science, University of Chinese Academy of Sciences（中国科学院大学工程科学学院）
Meshless solutions of PDE inverse problems on irregular geometries
School of Engineering and Applied Sciences, Harvard University, Cambridge MA 02138（工程与应用科学学院，哈佛大学，康科德马萨诸塞州02138）；School of Engineering（工程学院）；Applied Sciences, Harvard University, Cambridge MA 02138（应用科学，哈佛大学，康科德马萨诸塞州02138）；Department of Physics, Harvard University, Cambridge MA 02138（物理系，哈佛大学，康科德马萨诸塞州02138）
Quantum-Informed Machine Learning for Predicting Spatiotemporal Chaos with Practical Quantum Advantage
Centre for Computational Science, University College London（伦敦大学学院计算科学中心）；Informatics Institute, University of Amsterdam（阿姆斯特丹大学信息学院）；Centre for Advanced Research Computing, University College London（伦敦大学学院高级计算研究中心）
PARCv2: Physics-aware Recurrent Convolutional Neural Networks for Spatiotemporal Dynamics Modeling
School of Data Science（数据科学学院）；University of Virginia（弗吉尼亚大学）；Department of Astronomy（天文学系）；Department of Mechanical Engineering（机械工程系）；University of Iowa（爱荷华大学）；School of Mechanical Engineering（机械工程学院）；Kyungpook National University（庆北国立大学）；Department of Mechanical and Aerospace Engineering（机械与航空航天工程系）
Many scientific phenomena are modeled by Partial Differential Equations (PDEs). The development of data gathering tools along with the advances in machine learning (ML) techniques have raised opportunities for data-driven identification of governing equations from experimentally observed data. We propose an ML method to discover the terms involved in the equation from two-dimensional spatiotemporal data. Robust and useful physical features are extracted from data samples to represent the specific behaviors imposed by each mathematical term in the equation. Compared to the previous models, this…
PI-MFM: Physics-informed multimodal foundation model for solving partial differential equations
Department of Statistics and Data Science, Yale University（统计与数据科学系，耶鲁大学）；Department of Applied Mathematics and Statistics, Johns Hopkins University（应用数学与统计学系，约翰霍普金斯大学）；Department of Applied Computational Mathematics and Statistics, University of Notre Dame（应用计算数学与统计学系，圣母大学）；Department of Mathematics, University of California Los Angeles（数学系，加州大学洛杉矶分校）；Department of Chemical and Environmental Engineering, Yale University（化学与环境工程系，耶鲁大学）
Inverse Problems for Partial Differential Equations with Jump Discontinuities in Coefficients via Two-Stage Physics-Informed Deep Learning and Statistical Mixture Models
School of Mathematics and Statistics（数学与统计学学院）；Northwestern Polytechnical University（西北工业大学）；Huazhong University of Science and Technology（华中科技大学）；Xiangtan University（湘潭大学）；Southern University of Science and Technology（南方科技大学）；MOE Key Laboratory for Complexity Science in Aerospace（航空航天复杂科学教育部重点实验室）；SandGold AI Research（SandGold AI研究院）
University of Tübingen（图宾根大学）；University of Stuttgart（斯图加特大学）
We introduce a compositional physics-aware FInite volume Neural Network (FINN) for learning spatiotemporal advection-diffusion processes. FINN implements a new way of combining the learning abilities of artificial neural networks with physical and structural knowledge from numerical simulation by modeling the constituents of partial differential equations (PDEs) in a compositional manner. Results on both one- and two-dimensional PDEs (Burgers', diffusion-sorption, diffusion-reaction, Allen--Cahn) demonstrate FINN's superior modeling accuracy and excellent out-of-distribution generalization abi…
The measured spatiotemporal response of various physical processes is utilized to infer the governing partial differential equations (PDEs). We propose SimultaNeous Basis Function Approximation and Parameter Estimation (SNAPE), a technique of parameter estimation of PDEs that is robust against high levels of noise nearly 100 %, by simultaneously fitting basis functions to the measured response and estimating the parameters of both ordinary and partial differential equations. The domain knowledge of the general multidimensional process is used as a constraint in the formulation of the optimizat…
TU Dresden（德累斯顿工业大学）；Center for Systems Biology Dresden, Max Planck Institute of Molecular（马克斯·普朗克分子研究所德累斯顿系统生物学中心）；Flatiron Institute（弗莱提荣研究所）
We present a statistical learning framework for robust identification of partial differential equations from noisy spatiotemporal data. Extending previous sparse regression approaches for inferring PDE models from simulated data, we address key issues that have thus far limited the application of these methods to noisy experimental data, namely their robustness against noise and the need for manual parameter tuning. We address both points by proposing a stability-based model selection scheme to determine the level of regularization required for reproducible recovery of the underlying PDE. This…
A finite element-based physics-informed operator learning framework for spatiotemporal partial differential equations on arbitrary domains
Graduate School of Science and Technology, Keio University（庆应义塾大学理工学研究科）；Institute of Applied Mechanics, RWTH Aachen University（亚琛工业大学应用力学研究所）；ACCESS e.V.（ACCESS协会）
