arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.09051cs.SEcs.AI
收藏

RuleFlow : 利用LLMs生成可重用的程序优化

RuleFlow : Generating Reusable Program Optimizations with LLMs

University of Illinois Urbana-Champaign, USA(伊利诺伊大学厄巴纳-香槟分校)

Avaljot Singh, Dushyant Bharadwaj, Stefanos Baziotis, Kaushik Varadharajan, Charith Mendis

AI总结:

RuleFlow通过结合LLMs与编译器技术,实现了高效的Pandas程序优化,达到新的SOTA水平。

摘要:

对Pandas程序进行优化是一个具有挑战性的问题。现有的系统和基于编译器的方法虽然可靠,但要么过于沉重,要么只支持有限的优化种类。相反,使用LLMs在每程序优化方法中可以合成非平凡的优化,但不可靠、昂贵且产出低。在本工作中,我们引入了一种混合方法,以三种阶段的方式工作,将发现与部署解耦,并通过一种新的桥梁连接。首先,它发现每程序优化(发现)。其次,它们被转换为通用的重写规则(桥梁)。最后,这些规则被纳入编译器中,可以在适用的地方自动应用,从而消除对LLMs的重复依赖(部署)。我们证明RuleFlow是PandasBench上的新状态-of-the-art(SOTA)Pandas优化框架,PandasBench是一个具有挑战性的Pandas基准,由Python笔记本组成。在这些笔记本中,我们实现了比Dias(之前的基于编译器的SOTA)快高达4.3倍,以及比Modin(之前的基于系统的方法的SOTA)快1914.9倍的加速。我们的代码可在https://github.com/ADAPT-uiuc/RuleFlow上获得。

英文摘要:

Optimizing Pandas programs is a challenging problem. Existing systems and compiler-based approaches offer reliability but are either heavyweight or support only a limited set of optimizations. Conversely, using LLMs in a per-program optimization methodology can synthesize nontrivial optimizations, but is unreliable, expensive, and offers a low yield. In this work, we introduce a hybrid approach that works in a 3-stage manner that decouples discovery from deployment and connects them via a novel bridge. First, it discovers per-program optimizations (discovery). Second, they are converted into generalised rewrite rules (bridge). Finally, these rules are incorporated into a compiler that can automatically apply them wherever applicable, eliminating repeated reliance on LLMs (deployment). We demonstrate that RuleFlow is the new state-of-the-art (SOTA) Pandas optimization framework on PandasBench, a challenging Pandas benchmark consisting of Python notebooks. Across these notebooks, we achieve a speedup of up to 4.3x over Dias, the previous compiler-based SOTA, and 1914.9x over Modin, the previous systems-based SOTA. Our code is available at https://github.com/ADAPT-uiuc/RuleFlow.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑