面向现实世界的编译器助手
Agentic Harness for Real-World Compilers
AI总结：
本文提出llvm-autofix，首个专为修复编译器bug设计的代理工具，通过专用工具和基准测试展示其在处理复杂编译器bug时的性能优势。
摘要：
编译器对现代计算至关重要，但修复编译器bug极具挑战性。尽管大型语言模型（LLM）的进步使自动化修复成为可能，但编译器bug的复杂性、跨领域专业知识需求以及稀疏、非描述性的bug报告，需要专门的工具。为此，我们引入llvm-autofix，首个代理助手，旨在帮助LLM代理理解并修复编译器bug。我们的重点是LLVM，最广泛使用的编译器基础设施之一。llvm-autofix的核心包括友好的LLVM工具、可重复的LLVM bug基准测试llvm-bench，以及专为修复LLVM bug设计的最小代理llvm-autofix-mini。评估显示，前沿模型在处理编译器bug时性能下降60%，而llvm-autofix-mini比最先进的方法高出约22%。这强调了像我们这样的专用助手在连接LLM和编译器工程方面的重要性。我们相信这项工作为提升LLM在复杂系统如编译器中的能力奠定了基础。GitHub: https://github.com/dtcxzyw/llvm-autofix
英文摘要：
Compilers are critical to modern computing, yet fixing compiler bugs is difficult. While recent large language model (LLM) advancements enable automated bug repair, compiler bugs pose unique challenges due to their complexity, deep cross-domain expertise requirements, and sparse, non-descriptive bug reports, necessitating compiler-specific harnesses. To bridge the gap, we introduce llvm-harness, the first harness designed to assist LLM agents in understanding and fixing compiler bugs. Our current focus is on the middle end of LLVM, one of the most widely used compiler infrastructures. Central to llvm-harness are agent-friendly LLVM tools, a benchmark llvm-bench of 334 reproducible LLVM middle-end bugs, and a tailored mini agent llvm-autofix-mini for fixing LLVM middle-end bugs automatically. We evaluate five frontier models and find that they exhibit a performance decline when tackling compiler bugs with the state-of-the-art agent. With llvm-harness' enhancement, their performance improves by 62%. Our specialized mini agent llvm-autofix-mini further outperforms the llvm-harness-enhanced state-of-the-art by 22%. This emphasizes the necessity for specialized harnesses like ours to assist LLMs in compiler engineering tasks. Despite promising results, our expert review also reveals several open challenges that remain when applying LLMs for compiler engineering tasks. GitHub: https://github.com/dtcxzyw/llvm-harness
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
