arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.14714cs.PLcs.LG
收藏

基于编译器生成反馈的大型语言模型

Compiler generated feedback for Large Language Models

Meta AI;Rice University(莱斯大学)

Dejan Grubisic, Chris Cummins, Volker Seeker, Hugh Leather

AI总结:

本文提出一种结合编译器反馈与大型语言模型的LLVM汇编代码优化新范式,通过将编译验证结果反馈给模型进行迭代优化,相比-Oz额外提升0.53%,但发现简单采样技术在多样本下性能更优。

摘要:

我们引入了一种由大型语言模型结合编译器反馈驱动的新型编译器优化范式,用于优化LLVM汇编的代码大小。模型以未优化的LLVM IR作为输入,输出优化后的IR、最佳优化pass以及未优化和优化后IR的指令计数。随后我们使用生成的优化pass编译输入,并评估预测的指令计数是否正确、生成的IR是否可编译,以及是否与编译后的代码相对应。我们将此反馈提供给LLM,并给予其另一次优化代码的机会。该方法在原始模型基础上,相比-Oz额外实现了0.53%的性能提升。然而,尽管通过反馈添加更多信息看似直观,但在给定10个或更多样本的情况下,简单的采样技术却能取得高得多的性能。

英文摘要:

We introduce a novel paradigm in compiler optimization powered by Large Language Models with compiler feedback to optimize the code size of LLVM assembly. The model takes unoptimized LLVM IR as input and produces optimized IR, the best optimization passes, and instruction counts of both unoptimized and optimized IRs. Then we compile the input with generated optimization passes and evaluate if the predicted instruction count is correct, generated IR is compilable, and corresponds to compiled code. We provide this feedback back to LLM and give it another chance to optimize code. This approach adds an extra 0.53% improvement over -Oz to the original model. Even though, adding more information with feedback seems intuitive, simple sampling techniques achieve much higher performance given 10 or more samples.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑