depyf:为机器学习研究人员打开 PyTorch 编译器的不透明黑盒
depyf: Open the Opaque Box of PyTorch Compiler for Machine Learning Researchers
AI总结:
针对 PyTorch 编译器对机器学习研究人员不透明的问题,提出非侵入式工具 depyf,通过反编译字节码实现源码级调试,助力研究人员深入理解编译器底层过程。
摘要:
PyTorch 2.x 引入了一款旨在加速深度学习程序的编译器。然而,对于机器学习研究人员而言,要完全发挥 PyTorch 编译器的潜力颇具挑战。该编译器在 Python 字节码层面运行,使其表现为一个不透明的黑盒。为解决此问题,我们引入了 depyf,一款旨在揭开 PyTorch 编译器内部工作机制的工具。depyf 将 PyTorch 生成的字节码反编译回等效的源代码,并建立内存中的代码对象与其磁盘上源代码副本之间的联系。此功能使用户能够使用调试器逐行单步执行源代码,从而增强其对底层过程的理解。值得注意的是,depyf 是非侵入式且用户友好的,其核心功能主要依赖于两个便捷的上下文管理器。该项目已公开提供,并被认可为 PyTorch 生态系统项目。
英文摘要:
PyTorch \texttt{2.x} introduces a compiler designed to accelerate deep learning programs. However, for machine learning researchers, adapting to the PyTorch compiler to full potential can be challenging. The compiler operates at the Python bytecode level, making it appear as an opaque box. To address this, we introduce \texttt{depyf}, a tool designed to demystify the inner workings of the PyTorch compiler. \texttt{depyf} decompiles bytecode generated by PyTorch back into equivalent source code, and establishes connections between in-memory code objects and their on-disk source code counterparts. This feature enables users to step through the source code line by line using debuggers, thus enhancing their understanding of the underlying processes. Notably, \texttt{depyf} is non-intrusive and user-friendly, primarily relying on two convenient context managers for its core functionality. The project is \href{https://github.com/thuml/depyf}{ openly available} and is recognized as a \href{https://pytorch.org/ecosystem/}{PyTorch ecosystem project}.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。