用可逆的嵌入式领域特定语言区分一切
Differentiate Everything with a Reversible Embeded Domain-Specific Language
摘要：
反向模式自动微分（AD）遭受具有太多空间开销来追溯用于反向传播的中间计算状态的问题。跟踪状态的传统方法称为检查点，它将中间状态存储到全局堆栈中，并通过堆栈弹出或重新计算来恢复状态。堆栈操作和重新计算的开销使得通用（而不是基于张量的）AD引擎无法满足许多工业需求。而不是检查点，我们建议使用反向计算来追溯状态，通过设计和实现一个可逆的编程eDSL，其中程序可以双向执行，而无需隐式堆栈操作。隐式堆栈操作的缺失使得程序与现有的编译器功能兼容，包括利用现有的优化通道和将代码编译为GPU内核。我们实现了稀疏矩阵运算和一些机器学习应用的AD，以表明我们的框架具有最先进的性能。
英文摘要：
Reverse-mode automatic differentiation (AD) suffers from the issue of having too much space overhead to trace back intermediate computational states for back-propagation. The traditional method to trace back states is called checkpointing that stores intermediate states into a global stack and restore state through either stack pop or re-computing. The overhead of stack manipulations and re-computing makes the general purposed (not tensor-based) AD engines unable to meet many industrial needs. Instead of checkpointing, we propose to use reverse computing to trace back states by designing and implementing a reversible programming eDSL, where a program can be executed bi-directionally without implicit stack operations. The absence of implicit stack operations makes the program compatible with existing compiler features, including utilizing existing optimization passes and compiling the code as GPU kernels. We implement AD for sparse matrix operations and some machine learning applications to show that our framework has the state-of-the-art performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
