AI应该优化你的代码吗？经典优化编译器与当前大型语言模型的比较研究
Should AI Optimize Your Code? A Comparative Study of Classical Optimizing Compilers Versus Current Large Language Models
AI总结：
本研究比较了经典优化编译器与大型语言模型在代码优化中的能力，发现LLMs虽具潜力但在大代码上易出错，CodeLlama-70B表现最佳，且将编译器策略融入提示可显著提升性能。
摘要：
传统优化编译器在适应现代软件系统日益增长的复杂性方面发挥了重要作用。当前架构中对高效并行编程的需求需要强大的优化技术。大型语言模型（LLMs）的出现引发了关于这些AI方法在彻底改变代码优化方法方面潜力的有趣问题。这项工作旨在回答编译器社区的一个重要问题：“AI驱动的模型能否彻底改变我们处理代码优化的方式？”。为了解决这个问题，我们对三个经典优化编译器和两个最近的大型语言模型进行了比较分析，评估它们在优化代码以实现最大效率方面的各自能力和局限性。此外，我们引入了一组具有挑战性的优化模式基准套件，以及一种用于评估LLMs生成代码的性能和正确性的自动机制。我们使用了三种不同的提示策略来评估LLMs的性能：简单指令（IP）、详细指令提示（DIP）和思维链（CoT）。一个关键发现是，虽然LLMs有潜力超越当前的优化编译器，但它们经常在大型代码上生成不正确的代码，这需要自动验证方法。此外，将编译器策略作为LLMs提示的一部分表达出来，可以显著提高其整体性能。我们在三个基准套件上的评估显示，CodeLlama-70B是更优的LLM，能够实现高达x1.75的加速。此外，CETUS是当前优化编译器中最好的，实现了1.67x的最大加速。我们还发现三种提示策略之间存在实质性差异。
英文摘要：
Traditional optimizing compilers have played an important role in adapting to the growing complexity of modern software systems. The need for efficient parallel programming in current architectures requires strong optimization techniques. The beginning of Large Language Models (LLMs) raises intriguing questions about the potential of these AI approaches to revolutionize code optimization methodologies. This work aims to answer an essential question for the compiler community: "Can AI-driven models revolutionize the way we approach code optimization?". To address this question, we present a comparative analysis between three classical optimizing compilers and two recent large language models, evaluating their respective abilities and limitations in optimizing code for maximum efficiency. In addition, we introduce a benchmark suite of challenging optimization patterns and an automatic mechanism for evaluating the performance and correctness of the code generated by LLMs. We used three different prompting strategies to evaluate the performance of the LLMs, Simple Instruction (IP), Detailed Instruction Prompting (DIP), and Chain of Thought (CoT). A key finding is that while LLMs have the potential to outperform current optimizing compilers, they often generate incorrect code on large code sizes, calling for automated verification methods. In addition, expressing a compiler strategy as part of the LLMs prompt substantially improves its overall performance. Our evaluation across three benchmark suites shows CodeLlama-70B as the superior LLM, capable of achieving speedups of up to x1.75. Additionally, CETUS is the best among the current optimizing compilers, achieving a maximum speedup of 1.67x. We also found substantial differences among the three prompting strategies.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
