arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2209.01895cs.MS
收藏

编译程序的向前模式自动微分

Forward-Mode Automatic Differentiation of Compiled Programs

Max Aehle, Johannes Blühdorn, Max Sagebaum, Nicolas R. Gauger

AI总结:

本文提出Derivgrind工具,通过Valgrind框架在机器码层面为编译程序添加向前模式自动微分,无需完整源代码即可对跨语言或部分闭源软件求导,代价是运行时间增加30至75倍。

摘要:

算法微分(AD)是一组为计算机实现的函数提供偏导数的技术。此类函数可以通过其源代码,或通过编译其源代码时产生的中间表示提供给最先进的AD工具。我们提出了新颖的AD工具Derivgrind,它为编译程序的机器码增加了向前模式AD逻辑。Derivgrind利用Valgrind插桩框架对机器码进行结构化访问,并使用影子内存工具来存储点值。只有在定义输入和输出变量的文件中才最多需要访问源代码。Derivgrind的多功能性以运行时间增加30到75倍为代价,这是在一个基于偏微分方程数值求解器的基准测试上测得的。我们广泛的回归测试套件的结果表明,Derivgrind在GCC和Clang编译的程序(包括一个Python解释器)上产生正确结果,只有少量例外。虽然我们提供了Derivgrind无法正确处理的情景列表,但几乎所有这些都是学术反例或源自高度优化的数学库。只要避免对这些进行微分,Derivgrind就可以以很少的集成工作应用于前所未有的广泛的跨语言或部分闭源软件。

英文摘要:

Algorithmic differentiation (AD) is a set of techniques that provide partial derivatives of computer-implemented functions. Such a function can be supplied to state-of-the-art AD tools via its source code, or via an intermediate representation produced while compiling its source code. We present the novel AD tool Derivgrind, which augments the machine code of compiled programs with forward-mode AD logic. Derivgrind leverages the Valgrind instrumentation framework for a structured access to the machine code, and a shadow memory tool to store dot values. Access to the source code is required at most for the files in which input and output variables are defined. Derivgrind's versatility comes at the price of scaling the run-time by a factor between 30 and 75, measured on a benchmark based on a numerical solver for a partial differential equation. Results of our extensive regression test suite indicate that Derivgrind produces correct results on GCC- and Clang-compiled programs, including a Python interpreter, with a small number of exceptions. While we provide a list of scenarios that Derivgrind does not handle correctly, nearly all of them are academic counterexamples or originate from highly optimized math libraries. As long as differentiating those is avoided, Derivgrind can be applied to an unprecedentedly wide range of cross-language or partially closed-source software with little integration efforts.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑