PopPy: 在Python复合AI应用中机会性地利用并行性
PopPy: Opportunistically Exploiting Parallelism in Python Agentic Workflows
AI总结:
本文提出PopPy系统,通过识别Python应用中调用外部组件的并行化机会,从而在复合AI应用的端到端延迟上实现6.4倍的加速,同时保持顺序程序语义。
摘要:
复合AI应用通过使用通用编程语言如Python调用ML模型的调用,广泛应用于软件工程和企业自动化等用户-facing任务,使其端到端延迟成为关键瓶颈。与传统应用不同,执行时间主要由外部组件主导,这些组件无法通过传统语言优化系统如优化编译器来处理。为了解决这个问题,我们开发了PopPy,一个能够发现Python应用中调用这些重型外部组件的并行化机会的系统,包括那些用于复合AI应用的组件。PopPy支持Python的一个非常表达性的片段,并且需要最小的开发者输入来发现并行性。它结合了提前编译器和运行时,解决了从Python应用中提取并行性的三个关键挑战:语言复杂性、动态调度和变量变异。在一组真实的复合AI应用上,PopPy在端到端执行时间上相比标准Python执行实现了高达6.4倍的加速,同时保持顺序程序语义。
英文摘要:
Agentic workflows, which compose calls to ML models using a general-purpose programming language like Python, are widely used for a variety of user-facing tasks, from software engineering to enterprise automation, making their end-to-end latency a critical bottleneck. To improve their latency, developers can either manually parallelize them or use restricted agentic frameworks that cannot express or exploit all forms of parallelization. We address this by developing PopPy, a system that parallelizes external calls, e.g., AI models, in agentic workflows written in a very expressive subset of Python. PopPy combines an ahead-of-time compiler with a runtime, addressing three key challenges in extracting parallelism from Python applications: language complexity, dynamic dispatch, and variable mutation. On a set of real-world agentic workflows, PopPy achieves up to $6.7\times$ speedups in end-to-end execution time, compared to standard Python execution and several agentic frameworks, while preserving the sequential program semantics.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。