使用重写策略实现高效函数式自动微分
Using Rewrite Strategies for Efficient Functional Automatic Differentiation
AI总结：
本文在函数式数组编程语言中利用重写规则和策略组合子实现基于对偶数的自动微分，以兼顾微分实现的优雅性与优化调度的简洁表达，并通过微基准测试初步验证了该方法的可行性。
摘要：
自动微分（AD）已成为机器学习中的主导技术。AD框架最初是为命令式语言使用磁带（tape）实现的。与此同时，函数式AD实现也得到了发展，通常基于对偶数（dual numbers），这些实现更接近微积分的形式化规范，因此更容易证明其正确性。但这些论文主要关注正确性而非效率。最近，有研究表明，通过适当的优化，基于对偶数的方法可以实现高效运行。优化高度依赖于顺序，因为一个优化可能使另一个优化成为可能。因此，对优化调度进行细粒度控制可能非常有用。一种方法将编译器优化表示为重写规则，其应用可以通过策略语言进行组合和控制。先前的工作描述了在函数式语言的编译器中使用项重写和策略来生成高性能代码。在这项工作中，我们在函数式数组编程语言中使用重写规则和策略组合子实现基于对偶数的自动微分，以进行优化。我们的目标是将使用对偶数进行微分的优雅性与使用策略语言简洁表达优化调度结合起来。我们给出了初步证据，表明该方法在微基准测试上具有可行性。
英文摘要：
Automatic Differentiation (AD) has become a dominant technique in ML. AD frameworks have first been implemented for imperative languages using tapes. Meanwhile, functional implementations of AD have been developed, often based on dual numbers, which are close to the formal specification of differentiation and hence easier to prove correct. But these papers have focussed on correctness not efficiency. Recently, it was shown how an approach using dual numbers could be made efficient through the right optimizations. Optimizations are highly dependent on order, as one optimization can enable another. It can therefore be useful to have fine-grained control over the scheduling of optimizations. One method expresses compiler optimizations as rewrite rules, whose application can be combined and controlled using strategy languages. Previous work describes the use of term rewriting and strategies to generate high-performance code in a compiler for a functional language. In this work, we implement dual numbers AD in a functional array programming language using rewrite rules and strategy combinators for optimization. We aim to combine the elegance of differentiation using dual numbers with a succinct expression of the optimization schedule using a strategy language. We give preliminary evidence suggesting the viability of the approach on a micro-benchmark.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
