并行对偶数反向模式自动微分
Parallel Dual-Numbers Reverse AD
AI总结：
本文借鉴线性分解优化对偶数反向模式自动微分，提出一种在标准函数式语言中高效实现且支持任务并行的算法，并提供了能微分大部分Haskell98的实际实现。
摘要：
正如对偶数前向模式自动微分（AD）将每个标量值与其切线值配对一样，对偶数反向模式AD试图使用类似简单的思想来实现反向AD：通过将每个标量值与反向传播函数配对。Brunel、Mazza和Pagani分析了其在高阶输入语言上的正确性和效率，但该分析使用了自定义的操作语义，尚不清楚其是否能被高效实现。我们借鉴他们使用线性分解来优化对偶数反向模式AD的方法，得到一个具有正确复杂度且在支持可变数组的标准函数式语言（如Haskell）中能高效实现的算法。除线性分解外，我们的优化步骤包含函数式编程社区中广为人知的思想。我们通过提供一个可对Haskell98大部分内容进行微分的实际实现来展示该技术的应用。以往关于对偶数反向AD的工作需要序列化来构建反向传递，而我们展示了可将该技术应用于任务并行源程序并生成任务并行导数计算。
英文摘要：
Where dual-numbers forward-mode automatic differentiation (AD) pairs each scalar value with its tangent value, dual-numbers reverse-mode AD attempts to achieve reverse AD using a similarly simple idea: by pairing each scalar value with a backpropagator function. Its correctness and efficiency on higher-order input languages have been analysed by Brunel, Mazza and Pagani, but this analysis used a custom operational semantics for which it is unclear whether it can be implemented efficiently. We take inspiration from their use of linear factoring to optimise dual-numbers reverse-mode AD to an algorithm that has the correct complexity and enjoys an efficient implementation in a standard functional language with support for mutable arrays, such as Haskell. Aside from the linear factoring ingredient, our optimisation steps consist of well-known ideas from the functional programming community. We demonstrate the use of our technique by providing a practical implementation that differentiates most of Haskell98. Where previous work on dual numbers reverse AD has required sequentialisation to construct the reverse pass, we demonstrate that we can apply our technique to task-parallel source programs and generate a task-parallel derivative computation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
