We introduce variational spectral learning (VSL), a machine learning framework for solving partial differential equations (PDEs) that operates directly in the coefficient space of spectral expansions. VSL offers a principled bridge between variational PDE theory, spectral discretization, and contemporary machine learning practice. The core idea is to recast a given PDE \[ \mathcal{L}u = f \quad \text{in} \quad Q=Ω\times(0,T), \] together with boundary and initial conditions, into differentiable space-time energies built from strong-form least-squares residuals and weak (Galerkin) formulations. The solution is represented as a finite spectral expansion \[ u_N(x,t)=\sum_{n=1}^{N} c_n\,ϕ_n(x,t), \] where $ϕ_n$ are tensor-product Chebyshev bases in space and time, with Dirichlet-satisfying spatial modes enforcing homogeneous boundary conditions analytically. This yields a compact linear parameterization in the coefficient vector $\mathbf{c}$, while all PDE complexity is absorbed into the variational energy. We show how to construct strong-form and weak-form space-time functionals, augment them with initial-condition and Tikhonov regularization terms, and minimize the resulting objective with gradient-based optimization. In practice, VSL is implemented in TensorFlow using automatic differentiation and Keras cosine-decay-with-restarts learning-rate schedules, enabling robust optimization of moderately sized coefficient vectors. Numerical experiments on benchmark elliptic and parabolic problems, including one- and two-dimensional Poisson, diffusion, and Burgers-type equations, demonstrate that VSL attains accuracy comparable to classical spectral collocation with Crank-Nicolson time stepping, while providing a differentiable objective suitable for modern optimization tooling.
We consider the discretization of elliptic boundary-value problems by variational physics-informed neural networks (VPINNs), in which test functions are continuous, piecewise linear functions on a triangulation of the domain. We define an a posteriori error estimator, made of a residual-type term, a loss-function term, and data oscillation terms. We prove that the estimator is both reliable and efficient in controlling the energy norm of the error between the exact and VPINN solutions. Numerical results are in excellent agreement with the theoretical predictions.
Tsinghua University（清华大学）；Massachusetts Institute of Technology（麻省理工学院）；KU Leuven（荷语鲁汶大学）
We propose a conservative energy method based on neural networks with subdomains for solving variational problems (CENN), where the admissible function satisfying the essential boundary condition without boundary penalty is constructed by the radial basis function (RBF), particular solution neural network, and general neural network. The loss term is the potential energy, optimized based on the principle of minimum potential energy. The loss term at the interfaces has the lower order derivative compared to the strong form PINN with subdomains. The advantage of the proposed method is higher eff…
An Introduction to Solving the Least-Squares Problem in Variational Data Assimilation
CERFACS（欧洲研究计算流体力学中心）；University of Potsdam（波茨坦大学）；School of Mathematical, Physical and Computational Sciences, University of Reading（雷丁大学数学、物理与计算科学学院）；National Centre for Earth Observation（国家地球观测中心）；Department of Mathematics and Statistics, University of Strathclyde（斯特拉斯克莱德大学数学与统计系）；Scientific Computing Department, Science & Technology Facilities Council, Rutherford Appleton Laboratory（卢瑟福阿普顿实验室科学计算部）；Eindhoven University of Technology（埃因霍温理工大学）
Freie Universität Berlin（柏林自由大学）；Zuse Institute Berlin（柏林祖斯研究所）；dida Datenschmiede GmbH（dida数据锻造有限责任公司）；University of Vienna（维也纳大学）
The combination of Monte Carlo methods and deep learning has recently led to efficient algorithms for solving partial differential equations (PDEs) in high dimensions. Related learning problems are often stated as variational formulations based on associated stochastic differential equations (SDEs), which allow the minimization of corresponding losses using gradient-based optimization methods. In respective numerical implementations it is therefore crucial to rely on adequate gradient estimators that exhibit low variance in order to reach convergence accurately and swiftly. In this article, we…
Deep Domain Decomposition Method for Solving the Variational Inequality Problems
School of Mathematics and Computing Science, Guilin University of Electronics Technology（桂林电子科技大学数学与计算科学学院）；Center for Applied Mathematics of Guangxi (GUET)（广西应用数学中心）
We develop innovative algorithms for solving the strong-constraint formulation of four-dimensional variational data assimilation in large-scale applications. We present a space-time decomposition approach that employs domain decomposition along both the spatial and temporal directions in the overlapping case and involves partitioning of both the solution and the operators. Starting from the global functional defined on the entire domain, we obtain a type of regularized local functionals on the set of subdomains providing the order reduction of both the predictive and the data assimilation mode…
In this paper, we propose and study neural network based methods for solutions of high-dimensional quadratic porous medium equation (QPME). Three variational formulations of this nonlinear PDE are presented: a strong formulation and two weak formulations. For the strong formulation, the solution is directly parameterized with a neural network and optimized by minimizing the PDE residual. It can be proved that the convergence of the optimization problem guarantees the convergence of the approximate solution in the $L^1$ sense. The weak formulations are derived following Brenier, Y., 2020, which…
We present a novel method for using Neural Networks (NNs) for finding solutions to a class of Partial Differential Equations (PDEs). Our method builds on recent advances in Neural Radiance Field research (NeRFs) and allows for a NN to converge to a PDE solution much faster than classic Physically Informed Neural Network (PINNs) approaches.
The accurate numerical solution of partial differential equations is a central task in numerical analysis allowing to model a wide range of natural phenomena by employing specialized solvers depending on the scenario of application. Here, we develop a variational approach for solving partial differential equations governing the evolution of high dimensional probability distributions. Our approach naturally works on the unbounded continuous domain and encodes the full probability density function through its variational parameters, which are adapted dynamically during the evolution to optimally…
A Proximal Decomposition Method for Solving Convex Variational Inverse Problems
UPMC Université Paris 06（巴黎第六大学）；Laboratoire Jacques-Louis Lions – UMR 7598（雅克-路易·利翁斯实验室）；Université Paris-Est（巴黎东大学）；Institut Gaspard Monge and UMR CNRS 8049（加斯帕尔·蒙日研究所）
Variational quantum Monte Carlo (VMC) combined with neural-network quantum states offers a novel angle of attack on the curse-of-dimensionality encountered in a particular class of partial differential equations (PDEs); namely, the real- and imaginary time-dependent Schrödinger equation. In this paper, we present a simple generalization of VMC applicable to arbitrary time-dependent PDEs, showcasing the technique in the multi-asset Black-Scholes PDE for pricing European options contingent on many correlated underlying assets.
Brown University（布朗大学）；Worcester Polytechnic Institute（伍斯特理工学院）；Pacific Northwest National Laboratory（太平洋西北国家实验室）
Physics-informed neural networks (PINNs) [31] use automatic differentiation to solve partial differential equations (PDEs) by penalizing the PDE in the loss function at a random set of points in the domain of interest. Here, we develop a Petrov-Galerkin version of PINNs based on the nonlinear approximation of deep neural networks (DNNs) by selecting the {\em trial space} to be the space of neural networks and the {\em test space} to be the space of Legendre polynomials. We formulate the \textit{variational residual} of the PDE using the DNN approximation by incorporating the variational form o…
Variational Geometry-aware Neural Network based Method for Solving High-dimensional Diffeomorphic Mapping Problems
Department of Mathematics, The Chinese University of Hong Kong, Shatin, Hong Kong（数学系，香港中文大学（深圳校区））；Department of Mathematics, University of British Columbia, Vancouver, V6T1Z2, BC, Canada（数学系，不列颠哥伦比亚大学）；Department of Mathematics, The Chinese University of Hong Kong, Shatin, N.T., Hong Kong（数学系，香港中文大学（深圳校区））
Solving high-dimensional Bayesian inverse problems (BIPs) with the variational inference (VI) method is promising but still challenging. The main difficulties arise from two aspects. First, VI methods approximate the posterior distribution using a simple and analytic variational distribution, which makes it difficult to estimate complex spatially-varying parameters in practice. Second, VI methods typically rely on gradient-based optimization, which can be computationally expensive or intractable when applied to BIPs involving partial differential equations (PDEs). To address these challenges, …
We propose a First-Order System Least Squares (FOSLS) method based on deep-learning for numerically solving second-order elliptic PDEs. The method we propose is capable of dealing with either variational and non-variational problems, and because of its meshless nature, it can also deal with problems posed in high-dimensional domains. We prove the $Γ$-convergence of the neural network approximation towards the solution of the continuous problem, and extend the convergence proof to some well-known related methods. Finally, we present several numerical examples illustrating the performance of our…
In backward error analysis, an approximate solution to an equation is compared to the exact solution to a nearby modified equation. In numerical ordinary differential equations, the two agree up to any power of the step size. If the differential equation has a geometric property then the modified equation may share it. In this way, known properties of differential equations can be applied to the approximation. But for partial differential equations, the known modified equations are of higher order, limiting applicability of the theory. Therefore, we study symmetric solutions of discretized par…
In this paper, we propose a tensor type of discretization and optimization process for solving high dimensional partial differential equations. First, we design the tensor type of trial function for the high dimensional partial differential equations. Based on the tensor structure of the trial functions, we can do the direct numerical integration of the approximate solution without the help of Monte-Carlo method. Then combined with the Ritz or Galerkin method, solving the high dimensional partial differential equation can be transformed to solve a concerned optimization problem. Some numerical…
This article presents several numerical techniques for solving Laplace equation. A numerical FORTRAN solver is developed to solve the 2D laplace equation. The numerical approaches implemented in the solver include Jacobi, Gauss-Siedel, Successive Over Relaxation, Alternating Direct Implicit and Multigrid methods. Detailed comparison between different numerical methods is presented and discussed
RUNNs: Ritz-Uzawa Neural Networks for Solving Variational Problems
Basque Center for Applied Mathematics（巴斯克应用数学中心）；Instituto de Matemáticas, Pontificia Universidad Católica de Valparaíso（天主教法科大学瓦帕莱索数学研究所）；Department of Mathematics, CUNEF Universidad（CUNEF大学数学系）；Curtin University（科廷大学）；Department of Mathematics, University of the Basque Country UPV/EHU（巴斯克国立大学数学系）；Ikerbasque: Basque Foundation for Science（伊克拉巴斯克：巴斯克科学基金会）；School of Mathematical Sciences, University of Nottingham（诺丁汉大学数学科学学院）
