基于机器学习的求解器在物理模拟和科学计算中引起了广泛关注,一个突出的例子是物理信息神经网络(PINNs)。然而,PINNs 常常难以求解高频和多尺度偏微分方程(PDEs),这可能是由于神经网络训练过程中的频谱偏差所致。为了解决这个问题,我们求助于高斯过程(GP)框架。为了灵活地捕获主导频率,我们用学生 t 混合或高斯混合来对 PDE 解的功率谱进行建模。我们应用逆傅里叶变换(通过维纳-辛钦定理)获得协方差函数。由高斯混合谱导出的协方差对应于已知的谱混合核。接下来,我们在对数域中估计混合权重,我们证明这等价于放置一个杰弗里斯先验。它自动诱导稀疏性,修剪多余的频率,并将剩余的频率调整到接近真实值。第三,为了实现大规模配点上的高效和可扩展计算,这对捕获高频至关重要,我们将配点放置在网格上,并在每个输入维度上乘以我们的协方差函数。我们使用 GP 条件均值来预测解及其导数,以便拟合边界条件和方程本身。因此,我们可以推导出协方差矩阵中的克罗内克积结构。我们利用克罗内克积性质和多线性代数来提高计算效率和可扩展性,而无需低秩近似。我们在系统实验中展示了我们方法的优势。代码发布于 \url{https://github.com/xuangu-fang/Gaussian-Process-Slover-for-High-Freq-PDE}。
英文摘要:
Machine learning based solvers have garnered much attention in physical simulation and scientific computing, with a prominent example, physics-informed neural networks (PINNs). However, PINNs often struggle to solve high-frequency and multi-scale PDEs, which can be due to spectral bias during neural network training. To address this problem, we resort to the Gaussian process (GP) framework. To flexibly capture the dominant frequencies, we model the power spectrum of the PDE solution with a student $t$ mixture or Gaussian mixture. We apply the inverse Fourier transform to obtain the covariance function (by Wiener-Khinchin theorem). The covariance derived from the Gaussian mixture spectrum corresponds to the known spectral mixture kernel. Next, we estimate the mixture weights in the log domain, which we show is equivalent to placing a Jeffreys prior. It automatically induces sparsity, prunes excessive frequencies, and adjusts the remaining toward the ground truth. Third, to enable efficient and scalable computation on massive collocation points, which are critical to capture high frequencies, we place the collocation points on a grid, and multiply our covariance function at each input dimension. We use the GP conditional mean to predict the solution and its derivatives so as to fit the boundary condition and the equation itself. As a result, we can derive a Kronecker product structure in the covariance matrix. We use Kronecker product properties and multilinear algebra to promote computational efficiency and scalability, without low-rank approximations. We show the advantage of our method in systematic experiments. The code is released at \url{https://github.com/xuangu-fang/Gaussian-Process-Slover-for-High-Freq-PDE}.
Tensor Gaussian Processes: Efficient Solvers for Nonlinear PDEs
Kahlert School of Computing, University of Utah(犹他大学卡勒特计算学院);Department of Mathematics, University of Kentucky(肯塔基大学数学系);Department of Computing and Mathematical Sciences, California Institute of Technology(加州理工学院计算与数学科学系)
Bilevel optimization for learning hyperparameters: Application to solving PDEs and inverse problems with Gaussian processes
Department of Mathematics, Cornell University(数学系,康奈尔大学);Department of Computing and Mathematical Sciences, California Institute of Technology(计算与数学科学系,加州理工学院)
MOD-Net: A Machine Learning Approach via Model-Operator-Data Network for Solving PDEs
Shanghai Jiao Tong University(上海交通大学);Shanghai Center for Brain Science and Brain-Inspired Technology(上海脑科学与类脑研究中心);Peking University(北京大学);AI for Science Institute(北京科学智能研究院)
Latent Neural Operator Pretraining for Solving Time-Dependent PDEs
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院)
QCPINN: Quantum-Classical Physics-Informed Neural Networks for Solving PDEs
Computational Science and Engineering Department, Informatics Institute Istanbul Technical University(信息学院理工学院计算科学与工程系);Scientific Computing, Rutherford Appleton Laboratory Science and Technology Facilities Council (STFC)(科学与技术设施委员会(STFC)科学计算部)
Solving high-dimensional parabolic PDEs using the tensor train format
Freie Universität Berlin(柏林自由大学);BTU Cottbus-Senftenberg(科特布斯-森夫滕贝格勃兰登堡工业大学);dida Datenschmiede GmbH(dida数据锻造有限公司);Technische Universität Berlin(柏林工业大学);Universität Potsdam(波茨坦大学)
Learning, Solving and Optimizing PDEs with TensorGalerkin: an efficient high-performance Galerkin assembly algorithm
ETH Zurich, Switzerland(苏黎世联邦理工学院,瑞士);Imperial College London, UK(伦敦帝国学院,英国);Northeastern University, USA(东北大学,美国);Renmin University of China, China(中国人民大学,中国)
RBF-MGN:Solving spatiotemporal PDEs with Physics-informed Graph Neural Network
College of Aerospace Science and Engineering, National University of Defense Technology(国防科技大学航天科学与工程学院);National Innovation Institute of Defense Technology, Chinese Academy of Military Science(中国人民解放军军事科学院国防科技创新研究院)
Solving high-dimensional Hamilton-Jacobi-Bellman PDEs using neural networks: perspectives from the theory of controlled diffusions and measures on path space
Universität Potsdam(波茨坦大学);Freie Universität Berlin(柏林自由大学);Brandenburgische Technische Universität Cottbus-Senftenberg(勃兰登堡工业大学科特布斯-森夫滕贝格)
Multiphysics Bench: Benchmarking and Investigating Scientific Machine Learning for Multiphysics PDEs
xLeaF Lab, Hong Kong University of Science and Technology (Guangzhou)(xLeaF实验室,香港科技大学(广州));National University of Singapore(新加坡国立大学);AI for Science, Shanghai AI Lab(AI for Science,上海人工智能实验室)
Convolutional-neural-operator-based transfer learning for solving PDEs
School of Mathematics(数学系);Key Laboratory of Maritime Intelligent Cyberspace Technology (Hohai University), Ministry of Education(航海智能信息空间技术重点实验室(河海大学))
TENG: Time-Evolving Natural Gradient for Solving PDEs With Deep Neural Nets Toward Machine Precision
Massachusetts Institute of Technology(麻省理工学院);NSF AI Institute for Artificial Intelligence and Fundamental Interactions(NSF人工智能与基础相互作用AI研究院);Harvard University(哈佛大学)
An analysis of the derivative-free loss method for solving PDEs
Department of Mathematics and Statistics, University at Albany, State University of New York(数学与统计学系,阿尔巴尼大学,纽约州立大学);Department of Mathematics, Dartmouth College(数学系,达特茅斯学院)
A fast direct solver based neural network for solving PDEs
Department of Computer Science & Engineering, International Institute of Information Technology Bangalore (IIIT-B), India(计算机科学与工程系,国际信息学院班加罗尔(IIIT-B),印度);Department of Data Science and Artificial Intelligence, International Institute of Information Technology Bangalore (IIIT-B), India(数据科学与人工智能系,国际信息学院班加罗尔(IIIT-B),印度)
Sinc Kolmogorov-Arnold network and its application for solving PDEs with singularities
Skolkovo Institute of Science and Technology(斯克洛夫科学与技术研究所);Southern University of Science and Technology(南方科技大学);International Center for Mathematics(国际数学中心);National Center for Applied Mathematics Shenzhen (NCAMS)(深圳应用数学中心)