arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2302.07867cs.SEcs.AIcs.LGcs.PF
收藏

学习提升性能的代码编辑

Learning Performance-Improving Code Edits

University of Pennsylvania(宾夕法尼亚大学);Carnegie Mellon University(卡内基梅隆大学);Google DeepMind(谷歌DeepMind);Google(谷歌)

Alexander Shypula, Aman Madaan, Yimeng Zeng, Uri Alon, Jacob Gardner, Milad Hashemi, Graham Neubig, Parthasarathy Ranganathan, Osbert Bastani, Amir Yazdanbakhsh

AI总结:

该研究提出将大型语言模型适配于高级程序优化,构建了含7.7万对C++提交的数据集和基于gem5的评估环境,通过检索提示、思维链及微调等策略,实现了超越人类程序员的加速效果。

摘要:

随着摩尔定律的衰退,优化程序性能已成为软件研究的重点。然而,由于难以理解代码的语义,诸如 API 和算法更改等高级优化仍然难以实现。与此同时,预训练大型语言模型(LLMs)在解决各种编程任务方面展现出了强大的能力。为此,我们引入了一个将 LLMs 适配于高级程序优化的框架。首先,我们整理了一个由人类程序员对超过 77,000 对 C++ 竞赛编程提交进行性能提升编辑的数据集,并附带了广泛的单元测试。一个主要挑战是在商用硬件上测量性能存在显著的可变性,这可能导致虚假的“改进”。为了隔离并可靠地评估程序优化的影响,我们设计了一个基于 gem5 全系统模拟器(学术界和工业界使用的事实上的模拟器)的环境。接下来,我们提出了广泛的代码优化适配策略;对于提示,这些包括基于检索的少样本提示和思维链,对于微调,这些包括基于性能的条件生成和基于自我博弈的合成数据增强。这些技术的结合在八次生成中实现了 6.86 的平均加速比,高于单个程序员的平均优化水平(3.66)。使用我们模型最快的生成结果,我们将数据集上可能实现的最快加速比的新上限设定为 9.64,而使用现有最快的人类提交为 9.56。

英文摘要:

With the decline of Moore's law, optimizing program performance has become a major focus of software research. However, high-level optimizations such as API and algorithm changes remain elusive due to the difficulty of understanding the semantics of code. Simultaneously, pretrained large language models (LLMs) have demonstrated strong capabilities at solving a wide range of programming tasks. To that end, we introduce a framework for adapting LLMs to high-level program optimization. First, we curate a dataset of performance-improving edits made by human programmers of over 77,000 competitive C++ programming submission pairs, accompanied by extensive unit tests. A major challenge is the significant variability of measuring performance on commodity hardware, which can lead to spurious "improvements." To isolate and reliably evaluate the impact of program optimizations, we design an environment based on the gem5 full system simulator, the de facto simulator used in academia and industry. Next, we propose a broad range of adaptation strategies for code optimization; for prompting, these include retrieval-based few-shot prompting and chain-of-thought, and for finetuning, these include performance-conditioned generation and synthetic data augmentation based on self-play. A combination of these techniques achieves a mean speedup of 6.86 with eight generations, higher than average optimizations from individual programmers (3.66). Using our model's fastest generations, we set a new upper limit on the fastest speedup possible for our dataset at 9.64 compared to using the fastest human submissions available (9.56).

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑