Keqin Liu
Based on a class of associative algebras with zero-divisors which are called real-like algebras by us, we introduce a way of defining automatic differentiation and present different ways of doing automatic differentiation to compute the first, the second and the third derivatives of a function exactly and simultaneously.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
继续探索
This article provides an overview of some of the mathematical principles of Automatic Differentiation (AD). In particular, we summarise different descriptions of the Forward Mode of AD, like the matrix-vector product based approach, the idea of lifting functions to the algebra of dual numbers, the method of Taylor series expansion on dual numbers and the application of the push-forward operator, and explain why they all reduce to the same actual chain of computations. We further give a short mathematical description of some methods of higher-order Forward AD and, at the end of this paper, brie…
Automatic differentiation in finite element stress updating
该研究将自动微分应用于有限元应力更新，对比解析微分实现岩土工程超塑性临界状态模型的效果，证实AD可简化应力积分实现，同时保持高精度，为计算力学代码开发提供便利。
Dual Numbers for Arbitrary Order Automatic Differentiation
本文提出DNAOAD，一种基于Fortran的自动微分框架，利用直接双数表示高效计算任意阶导数，解决传统方法内存消耗大、可扩展性差的问题。
Scalable Analysis and Design Using Automatic Differentiation
本文提出将自动微分与有限元算子分解结合，在积分点级别局部应用，以求解PDE约束优化和非线性有限元分析中的导数与雅可比矩阵，该方法高效、可扩展且非侵入式，并在大规模问题上验证了性能。
Many engineering problems involve learning hidden dynamics from indirect observations, where the physical processes are described by systems of partial differential equations (PDE). Gradient-based optimization methods are considered scalable and efficient to learn hidden dynamics. However, one of the most time-consuming and error-prone tasks is to derive and implement the gradients, especially in systems of PDEs where gradients from different systems must be correctly integrated together. To that purpose, we present a novel technique, called intelligent automatic differentiation (IAD), to leve…
Challenges in automatic differentiation and numerical integration in physics-informed neural networks modelling
本文研究 PINNs 中自动微分与数值积分的隐蔽精度问题，通过三个微分方程用例分析双精度是否足够，并评估多种求积方法以指导方法选择。
Scalable higher-order nonlinear solvers via higher-order automatic differentiation
本文通过Taylor模式自动微分高效计算高阶导数，提出并实现了高阶收敛的非线性求解器，发现二阶Halley方法最具价值，将其推广至非线性方程组及刚性ODE求解器中，证明其在大规模病态问题上优于Newton方法。
Automatic Differentiation: Inverse Accumulation Mode
本文提出一种自动求导方法，通过逆累积模式高效计算雅可比-逆向量和雅可比-逆转置向量乘积，突破传统计算限制，为牛顿法等数值计算提供新途径。
Determining physical properties inside an object without access to direct measurements of target regions can be formulated as a specific type of \textit{inverse problem}. One of such problems is applied in \textit{Electrical Impedance Tomography} (EIT). In general, EIT can be posed as a minimization problem and solved by iterative methods, which require knowledge of derivatives of the objective function. In practice, this can be challenging because analytical closed-form solutions for them are hard to derive and implement efficiently. In this paper, we study the effectiveness of \textit{automa…
Automatic differentiation (AD) is an ensemble of techniques that allow to evaluate accurate numerical derivatives of a mathematical function expressed in a computer programming language. In this paper we use AD for stating and solving solid mechanics problems. Given a finite element discretization of the domain, we evaluate the free energy of the solid as the integral of its strain energy density, and we make use of AD for directly obtaining the residual force vector and the tangent stiffness matrix of the problem, as the gradient and the Hessian of the free energy respectively. The result is …
FISTA-Condat-Vu: Automatic Differentiation for Hyperparameter Learning in Variational Models
针对工业CT场景下非光滑变分模型的正则化超参数估计问题，提出结合FISTA与Condat-Vu优势、基于低内存自动微分的双层学习求解策略，经工业断层数据验证有效且开源实现。
In this note, we report the back propagation formula for complex valued singular value decompositions (SVD). This formula is an important ingredient for a complete automatic differentiation(AD) infrastructure in terms of complex numbers, and it is also the key to understand and utilize AD in tensor networks.
A Novel Perspective Process Simulation Framework Based on Automatic Differentiation
本研究提出基于自动微分的热力学计算框架，在不改变算法逻辑下获取精确导数，显著提升了过程模拟的收敛性与计算效率，并展现出更强的通用性。
We present a novel class of approximations for variational losses, being applicable for the training of physics-informed neural nets (PINNs). The loss formulation reflects classic Sobolev space theory for partial differential equations and their weak formulations. The loss computation rests on an extension of Gauss-Legendre cubatures, we term Sobolev cubatures, replacing automatic differentiation (A.D.). We prove the runtime complexity of training the resulting Soblev-PINNs (SC-PINNs) to be less than required by PINNs relying on A.D. On top of one-to-two order of magnitude speed-up the SC-PINN…
A Common Interface for Automatic Differentiation
针对含大量自定义代码的科学机器学习任务，研发出Julia包DifferentiationInterface.jl，为十余个自动微分后端提供通用接口，通过内置准备机制摊销计算以发挥各后端优势，支持稀疏性处理等功能且不增加用户负担。
A critical step in topology optimization (TO) is finding sensitivities. Manual derivation and implementation of the sensitivities can be quite laborious and error-prone, especially for non-trivial objectives, constraints and material models. An alternate approach is to utilize automatic differentiation (AD). While AD has been around for decades, and has also been applied in TO, wider adoption has largely been absent. In this educational paper, we aim to reintroduce AD for TO, and make it easily accessible through illustrative codes. In particular, we employ JAX, a high-performance Python libra…
Automatic Backward Differentiation for American Monte-Carlo Algorithms (Conditional Expectation)
本文推导了包含条件期望算子的算法的反向微分方法，应用于美式金融衍生品估值，提出利用条件期望的线性组合和期望微分性质简化实现。
To target challenges in differentiable optimization we analyze and propose strategies for derivatives of the Matérn kernel with respect to the smoothness parameter. This problem is of high interest in Gaussian processes modelling due to the lack of robust derivatives of the modified Bessel function of second kind with respect to order. In the current work we focus on newly identified series expansions for the modified Bessel function of second kind valid for complex orders. Using these expansions we obtain highly accurate results using the complex step method. Furthermore, we show that the eva…
Shape calculus and automatic differentiation for multi-phase level-set topology optimisation with unfitted finite elements
本文提出形状微积分与可扩展自动形状微分框架，用于非拟合多相拓扑优化，经验证高精度，支持大规模并行，并应用于各向异性扩散、线弹性和流固耦合问题。
We describe and demonstrate an optimization-based x-ray image reconstruction framework called Adorym. Our framework provides a generic forward model, allowing one code framework to be used for a wide range of imaging methods ranging from near-field holography to and fly-scan ptychographic tomography. By using automatic differentiation for optimization, Adorym has the flexibility to refine experimental parameters including probe positions, multiple hologram alignment, and object tilts. It is written with strong support for parallel processing, allowing large datasets to be processed on high-per…
A C++ implementation of the discrete adjoint sensitivity analysis method for explicit adaptive Runge-Kutta methods enabled by automatic adjoint differentiation and SIMD vectorization
本文实现了一个面向自适应显式 Runge-Kutta 方法的 C++ 离散伴随灵敏度分析库，结合 AAD 与 SIMD 并行计算向量-Jacobian 乘积，性能略优于现有同类软件。
Understanding Automatic Differentiation Pitfalls
本文系统分类了自动微分（AD）的常见陷阱，通过混沌、离散化等示例说明，并回顾调试技术，以帮助用户避免错误并合理预期。
Mini-symposium on automatic differentiation and its applications in the financial industry
本文探讨自动微分在金融行业中的应用，介绍如何通过反向自动微分算法计算金融敏感性，并指出需要专门工具和理论支持以应对复杂计算需求。
Optimized Sparse Matrix Operations for Reverse Mode Automatic Differentiation
本研究为PyTorch实现了基于CSR的稀疏矩阵封装，支持CUDA加速和自动微分，并在优化问题中展示了其性能优势。
Automatic differentiation for performing the Cauchy-Kovalevskaya procedure in Lax-Wendroff type discretizations
本文引入自动微分（AD）执行Lax-Wendroff方法中的Cauchy-Kowalewski过程，实现任意阶数、无需雅可比矩阵且问题无关的预测步计算，数值实验验证了方法的精度和正性保持。
Level-set topology optimisation with unfitted finite elements and automatic shape differentiation
本文提出了一种非拟合自动形状微分方法，用于解决拓扑优化问题，通过Julia实现，适用于无结构网格，并避免了域边界处的材料属性平滑问题。
In scientific computing and machine learning applications, matrices and more general multidimensional arrays (tensors) can often be approximated with the help of low-rank decompositions. Since matrices and tensors of fixed rank form smooth Riemannian manifolds, one of the popular tools for finding low-rank approximations is to use Riemannian optimization. Nevertheless, efficient implementation of Riemannian gradients and Hessians, required in Riemannian optimization algorithms, can be a nontrivial task in practice. Moreover, in some cases, analytic formulas are not even available. In this pape…
Bringing PDEs to JAX with forward and reverse modes automatic differentiation
本文通过扩展JAX自动微分库并接入Firedrake有限元库，利用切线线性方程和伴随方程实现偏微分方程求解器的前向和反向模式自动微分，使其能与任意可微程序高效组合。
asdex: Automatic Sparse Differentiation in JAX
asdex是JAX生态系统中首个自动稀疏微分工具包，通过四步流程利用导数矩阵的稀疏性，减少AD过程数量，为相关组件提供稀疏替代实现。
Derivative Computation in PINNs: Automatic Differentiation, Finite Differences and Beyond
该研究对比PINNs中FD与AD的导数计算性能，发现FD精度相当且速度更快、内存占用更少，其随机变体在稳态问题上优于AD，还解决了含样本间依赖架构的逐样本导数计算问题。
A tutorial on automatic differentiation with complex numbers
针对复数自动微分相关文档匮乏的问题，本教程借助线性代数推导实现复数版雅可比向量积等，综述复数前向与反向模式自动微分，指导自定义梯度传播规则的实现。
Automatic Differentiation from Scratch: How PyTorch Computes Gradients in Physics-Informed Neural Networks
研究PyTorch在物理信息神经网络中计算梯度的方式，借助1-3-3-1多层感知器和特定初值问题，追踪前向传播计算图、反向遍历及图中图机制，验证伴随值并联系相关框架与积，明确其计算梯度的完整流程。
We propose an algorithm to compute the $C^\infty$-ring structure of arbitrary Weil algebra. It allows us to do some analysis with higher infinitesimals numerically and symbolically. To that end, we first give a brief description of the (Forward-mode) automatic differentiation (AD) in terms of $C^\infty$-rings. The notion of a $C^\infty$-ring was introduced by Lawvere and used as the fundamental building block of smooth infinitesimal analysis and synthetic differential geometry. We argue that interpreting AD in terms of $C^\infty$-rings gives us a unifying theoretical framework and modular ways…
扫码添加微信好友，提出您的宝贵建议 👇
💡 备注请填写：网站反馈
发表机构由 AI 从论文文本抽取，如发现错误，欢迎反馈修正建议。