LLVM is a widely used compiler infrastructure whose scale and complexity make issue resolution labor-intensive and challenging. Although large language models (LLMs) have recently achieved remarkable success in issue resolution, their effectiveness on complex system-level LLVM compiler remains largely unexplored. To address this gap, we introduce LLVM-Bench, the first large-scale benchmark for LLVM issue resolution, containing 423 real-world, validated tasks collected from the LLVM project. We further develop LLVM-Gym, a scalable evaluation platform that automates issue reproduction, patch application, compiler building, and test execution. Using LLVM-Bench and LLVM-Gym, we conduct a comprehensive study of four representative LLMs, six retrieval configurations, and three agents. Our results show that current LLM-based issue resolution techniques remain limited on LLVM-Bench, with patch invalidity and build failures as the dominant failure modes. We further reveal a strong complementarity among different LLMs and agents, motivating LLVM-Ens, a lightweight ensemble approach that expands the patch space through integrating the patches generated by diverse techniques, filters incorrect and redundant candidates, and identifies the most promising solution. Our results show that LLVM-Ens achieves a resolution rate of up to 21.99%, further improving LLVM issue resolution.
ECCO: Evidence-Driven Causal Reasoning for Compiler Optimization
Institute of Software at Chinese Academy of Sciences(中国科学院软件研究所);Hangzhou Institute for Advanced Study at University of Chinese Academy of Sciences(中国科学院大学杭州高等研究 institute);University of Chinese Academy of Sciences(中国科学院大学)
Synergy-Guided Compiler Auto-Tuning of Nested LLVM Pass Pipelines
Hangzhou Institute for Advanced Study at UCAS(中国科学院大学杭州高等研究院);Institute of Software, Chinese Academy of Sciences(中国科学院软件研究所);University of Chinese Academy of Sciences(中国科学院大学)
LPO: Discovering Missed Peephole Optimizations with Large Language Models
Cheriton School of Computer Science, University of Waterloo(滑铁卢大学 Cheriton 计算机科学学院);Department of Software Systems & Cybersecurity, Monash University(蒙纳士大学软件系统与网络安全系)