arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2408.12159cs.SEcs.AIcs.CL
收藏

基于搜索的大语言模型代码优化

Search-Based LLMs for Code Optimization

The Chinese University of Hong Kong(香港中文大学);Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳));Peng Cheng Laboratory(鹏城实验室)

Shuzheng Gao, Cuiyun Gao, Wenchao Gu, Michael Lyu

AI总结:

提出SBLLM框架,通过整合LLMs与进化搜索,采用基于执行的样本选择、自适应优化模式检索和遗传算子启发的思维链提示,实现代码优化的迭代精炼与发现,解决一步生成范式的局限。

摘要:

开发人员编写的代码通常存在效率问题,并包含各种性能漏洞。这些低效问题促使了对代码优化的自动化重构方法的研究。早期的代码优化研究采用基于规则的方法,并专注于特定的低效问题,这不仅耗费人力,而且存在覆盖率低的问题。最近的工作将此任务视为序列生成问题,并求助于深度学习(DL)技术,如大语言模型(LLMs)。这些方法通常提示LLMs直接生成优化代码。尽管这些方法展现了最先进的性能,但这种一步生成范式难以实现最优解。首先,诸如组合优化等复杂优化方法很难被LLMs捕捉。其次,一步生成范式在精确注入LLMs进行有效代码优化所需的知识方面面临挑战,导致代码优化不足。为了解决这些问题,我们提出从搜索视角对该任务进行建模,并提出一个名为SBLLM的基于搜索的LLMs框架,该框架能够迭代地精炼和发现改进的优化方法。SBLLM将LLMs与进化搜索协同整合,包含三个关键组件:1)基于执行的代表性样本选择部分,评估每个现有优化代码的适应度,并优先考虑有前景的代码以引导改进代码的生成;2)自适应优化模式检索部分,将目标优化模式注入模型,以引导LLMs纠正并逐步增强其优化方法;3)受遗传算子启发的思维链提示部分,帮助LLMs结合不同的优化方法并生成改进的优化方法。

英文摘要:

The code written by developers usually suffers from efficiency problems and contain various performance bugs. These inefficiencies necessitate the research of automated refactoring methods for code optimization. Early research in code optimization employs rule-based methods and focuses on specific inefficiency issues, which are labor-intensive and suffer from the low coverage issue. Recent work regards the task as a sequence generation problem, and resorts to deep learning (DL) techniques such as large language models (LLMs). These methods typically prompt LLMs to directly generate optimized code. Although these methods show state-of-the-art performance, such one-step generation paradigm is hard to achieve an optimal solution. First, complex optimization methods such as combinatorial ones are hard to be captured by LLMs. Second, the one-step generation paradigm poses challenge in precisely infusing the knowledge required for effective code optimization within LLMs, resulting in under-optimized code.To address these problems, we propose to model this task from the search perspective, and propose a search-based LLMs framework named SBLLM that enables iterative refinement and discovery of improved optimization methods. SBLLM synergistically integrate LLMs with evolutionary search and consists of three key components: 1) an execution-based representative sample selection part that evaluates the fitness of each existing optimized code and prioritizes promising ones to pilot the generation of improved code; 2) an adaptive optimization pattern retrieval part that infuses targeted optimization patterns into the model for guiding LLMs towards rectifying and progressively enhancing their optimization methods; and 3) a genetic operator-inspired chain-of-thought prompting part that aids LLMs in combining different optimization methods and generating improved optimization methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑