arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.06752cs.MScs.LGcs.NAmath.NA

复数自动微分教程

A tutorial on automatic differentiation with complex numbers

Technical University of Denmark（丹麦技术大学）

Nicholas Krämer

阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对复数自动微分相关文档匮乏的问题，本教程借助线性代数推导实现复数版雅可比向量积等，综述复数前向与反向模式自动微分，指导自定义梯度传播规则的实现。

摘要：

自动微分应用十分广泛，但关于其在复数运算中如何工作的文档却极少，仅提及“$\boldsymbol{\rm{C}}^d$中的导数”等价于“$\boldsymbol{\rm{R}}^{2d}$中的导数”，充其量也只是浅显提及Wirtinger微积分（维尔丁格微积分）。遗憾的是，一旦我们需要推导自定义梯度规则，例如为了避免对高成本线性代数函数或微分方程模拟器进行“穿透式”求导，$\boldsymbol{\rm{C}}^d \boldsymbol{\rm{R}}^{2d}$的等价关系就不再够用。为弥补这类文档缺失，本文综述了复数的前向模式与反向模式自动微分，涵盖Wirtinger导数、修正链式法则以及不同的梯度约定，同时明确避开全纯性与柯西-黎曼方程（二者的限制过强）。确切地说，我们将几乎完全借助线性代数推导、阐释并实现复数版本的雅可比向量积与向量雅可比积，无需依赖复分析或微分几何。本教程呼吁用户与开发者在实现自定义梯度传播规则时重视复数值，本文阐述了具体实现方法。

英文摘要：

Automatic differentiation is everywhere, but there exists only minimal documentation of how it works in complex arithmetic beyond stating "derivatives in $\mathbb{C}^d$" $\cong$ "derivatives in $\mathbb{R}^{2d}$" and, at best, shallow references to Wirtinger calculus. Unfortunately, the equivalence $\mathbb{C}^d \cong \mathbb{R}^{2d}$ becomes insufficient as soon as we need to derive custom gradient rules, e.g., to avoid differentiating "through" expensive linear algebra functions or differential equation simulators. To combat such a lack of documentation, this article surveys forward- and reverse-mode automatic differentiation with complex numbers, covering topics such as Wirtinger derivatives, a modified chain rule, and different gradient conventions while explicitly avoiding holomorphicity and the Cauchy--Riemann equations (which would be far too restrictive). To be precise, we will derive, explain, and implement a complex version of Jacobian-vector and vector-Jacobian products almost entirely with linear algebra without relying on complex analysis or differential geometry. This tutorial is a call to action, for users and developers alike, to take complex values seriously when implementing custom gradient propagation rules -- the manuscript explains how.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑