Automatic differentiation (AD) is an ensemble of techniques that allow to evaluate accurate numerical derivatives of a mathematical function expressed in a computer programming language. In this paper we use AD for stating and solving solid mechanics problems. Given a finite element discretization of the domain, we evaluate the free energy of the solid as the integral of its strain energy density, and we make use of AD for directly obtaining the residual force vector and the tangent stiffness matrix of the problem, as the gradient and the Hessian of the free energy respectively. The result is a remarkable simplification in the statement and the solution of complex problems involving non trivial constraints systems and both geometrical and material non linearities. Together with the continuum mechanics theoretical basis, and with a description of the specific AD technique adopted, the paper illustrates the solution of a number of solid mechanics problems, with the aim of presenting a convenient numerical implementation approach, made easily available by recent programming languages, to the solid mechanics community.
Based on a class of associative algebras with zero-divisors which are called real-like algebras by us, we introduce a way of defining automatic differentiation and present different ways of doing automatic differentiation to compute the first, the second and the third derivatives of a function exactly and simultaneously.
This article provides an overview of some of the mathematical principles of Automatic Differentiation (AD). In particular, we summarise different descriptions of the Forward Mode of AD, like the matrix-vector product based approach, the idea of lifting functions to the algebra of dual numbers, the method of Taylor series expansion on dual numbers and the application of the push-forward operator, and explain why they all reduce to the same actual chain of computations. We further give a short mathematical description of some methods of higher-order Forward AD and, at the end of this paper, brie…
Many engineering problems involve learning hidden dynamics from indirect observations, where the physical processes are described by systems of partial differential equations (PDE). Gradient-based optimization methods are considered scalable and efficient to learn hidden dynamics. However, one of the most time-consuming and error-prone tasks is to derive and implement the gradients, especially in systems of PDEs where gradients from different systems must be correctly integrated together. To that purpose, we present a novel technique, called intelligent automatic differentiation (IAD), to leve…
Determining physical properties inside an object without access to direct measurements of target regions can be formulated as a specific type of \textit{inverse problem}. One of such problems is applied in \textit{Electrical Impedance Tomography} (EIT). In general, EIT can be posed as a minimization problem and solved by iterative methods, which require knowledge of derivatives of the objective function. In practice, this can be challenging because analytical closed-form solutions for them are hard to derive and implement efficiently. In this paper, we study the effectiveness of \textit{automa…
In this note, we report the back propagation formula for complex valued singular value decompositions (SVD). This formula is an important ingredient for a complete automatic differentiation(AD) infrastructure in terms of complex numbers, and it is also the key to understand and utilize AD in tensor networks.
CASUS - Center for Advanced System Understanding（先进系统理解中心）；Helmholtz-Zentrum Dresden-Rossendorf e.V. (HZDR)（亥姆霍兹德累斯顿罗森多夫研究中心）
We present a novel class of approximations for variational losses, being applicable for the training of physics-informed neural nets (PINNs). The loss formulation reflects classic Sobolev space theory for partial differential equations and their weak formulations. The loss computation rests on an extension of Gauss-Legendre cubatures, we term Sobolev cubatures, replacing automatic differentiation (A.D.). We prove the runtime complexity of training the resulting Soblev-PINNs (SC-PINNs) to be less than required by PINNs relying on A.D. On top of one-to-two order of magnitude speed-up the SC-PINN…
LVMT, ENPC, Institut Polytechnique de Paris, Univ Gustave Eiffel（巴黎高等路桥学院，巴黎理工学院，古斯塔夫·埃菲尔大学，LVMT实验室）；BIFOLD – Berlin Institute for the Foundations of Learning and Data（柏林学习与数据基础研究所（BIFOLD））；Machine Learning Group, Technical University of Berlin（柏林工业大学机器学习组）
A critical step in topology optimization (TO) is finding sensitivities. Manual derivation and implementation of the sensitivities can be quite laborious and error-prone, especially for non-trivial objectives, constraints and material models. An alternate approach is to utilize automatic differentiation (AD). While AD has been around for decades, and has also been applied in TO, wider adoption has largely been absent. In this educational paper, we aim to reintroduce AD for TO, and make it easily accessible through illustrative codes. In particular, we employ JAX, a high-performance Python libra…
To target challenges in differentiable optimization we analyze and propose strategies for derivatives of the Matérn kernel with respect to the smoothness parameter. This problem is of high interest in Gaussian processes modelling due to the lack of robust derivatives of the modified Bessel function of second kind with respect to order. In the current work we focus on newly identified series expansions for the modified Bessel function of second kind valid for complex orders. Using these expansions we obtain highly accurate results using the complex step method. Furthermore, we show that the eva…
We describe and demonstrate an optimization-based x-ray image reconstruction framework called Adorym. Our framework provides a generic forward model, allowing one code framework to be used for a wide range of imaging methods ranging from near-field holography to and fly-scan ptychographic tomography. By using automatic differentiation for optimization, Adorym has the flexibility to refine experimental parameters including probe positions, multiple hologram alignment, and object tilts. It is written with strong support for parallel processing, allowing large datasets to be processed on high-per…
A C++ implementation of the discrete adjoint sensitivity analysis method for explicit adaptive Runge-Kutta methods enabled by automatic adjoint differentiation and SIMD vectorization
本文实现了一个面向自适应显式 Runge-Kutta 方法的 C++ 离散伴随灵敏度分析库，结合 AAD 与 SIMD 并行计算向量-Jacobian 乘积，性能略优于现有同类软件。
Marchuk Institute of Numerical Mathematics of the Russian Academy of Sciences（俄罗斯科学院马尔丘克计算数学研究所）；HSE University（高等经济学院）；Skolkovo Institute of Science and Technology（斯科尔科沃科学技术学院）
In scientific computing and machine learning applications, matrices and more general multidimensional arrays (tensors) can often be approximated with the help of low-rank decompositions. Since matrices and tensors of fixed rank form smooth Riemannian manifolds, one of the popular tools for finding low-rank approximations is to use Riemannian optimization. Nevertheless, efficient implementation of Riemannian gradients and Hessians, required in Riemannian optimization algorithms, can be a nontrivial task in practice. Moreover, in some cases, analytic formulas are not even available. In this pape…
BIFOLD – Berlin Institute for the Foundations of Learning and Data（柏林学习与数据基础研究所）；LVMT, ENPC, Institut Polytechnique de Paris, Univ Gustave Eiffel（法国国立路桥学校，巴黎理工学院，古斯塔夫·埃菲尔大学）；Machine Learning Group, Technical University of Berlin（柏林工业大学机器学习组）
We propose an algorithm to compute the $C^\infty$-ring structure of arbitrary Weil algebra. It allows us to do some analysis with higher infinitesimals numerically and symbolically. To that end, we first give a brief description of the (Forward-mode) automatic differentiation (AD) in terms of $C^\infty$-rings. The notion of a $C^\infty$-ring was introduced by Lawvere and used as the fundamental building block of smooth infinitesimal analysis and synthetic differential geometry. We argue that interpreting AD in terms of $C^\infty$-rings gives us a unifying theoretical framework and modular ways…
