手动软件优化是否已成为过去式？

中文摘要 AI 辅助

科学软件日益需要处理更大的数据集，同时保持可接受的执行时间。软件优化传统上需要大量编程、算法和数值方法方面的专业知识。大型语言模型（LLM）的最新进展为自动化这一过程中的大部分工作提供了可能性。我们研究了基于LLM的智能体能否自主实现科学软件性能的显著提升，包括那些已经过人类开发者广泛优化的成熟实现。我们让一个基于LLM的智能体针对三个计算问题优化软件：t-SNE、单样本基因集富集分析（ssGSEA）和图计数。人类定义了范围、正确性标准和验证机制，之后智能体自主工作，在某些情况下持续数小时。代码维护者审查了每个生成的实现并验证了其正确性。优化后的实现在所有测试配置中均更快，比现有最快工具快达两个数量级。改进包括底层代码优化、数学重构以及一种全新的图计数算法。软件优化可以越来越多地委托给自主智能体，人类的角色从实施优化转变为决定优化哪些软件、定义目标、提供验证机制以及确保最终软件的正确性。对于范围明确、可验证的问题，我们认为手动软件优化可能已成为过去式。

英文摘要

Scientific software is increasingly required to process larger datasets while maintaining acceptable execution times. Software optimization traditionally requires substantial expertise in programming, algorithms, and numerical methods. Recent advances in large language models (LLMs) offer the possibility of automating much of this process. We investigate whether LLM-based agents can autonomously achieve substantial performance improvements in scientific software, including mature implementations that have already been extensively optimized by human developers. We tasked an LLM-based agent with optimizing software for three computational problems: t-SNE, single-sample gene set enrichment analysis (ssGSEA), and graphlet counting. Humans defined the scope, correctness criteria, and a verification mechanism, after which the agent worked autonomously, in some cases for several hours. Code maintainers reviewed each resulting implementation and verified its correctness. The optimized implementations were faster in all tested configurations, by up to two orders of magnitude over the fastest existing tools. The improvements included low-level code optimizations, mathematical reformulations, and an entirely new algorithm for graphlet counting. Software optimization can increasingly be delegated to autonomous agents, with the human role shifting from implementing optimizations to deciding which software to optimize, defining objectives, providing verification mechanisms, and ensuring the correctness of the final software. For well-scoped, verifiable problems, we argue that manual software optimization may be a thing of the past.

发表机构

• University of Ljubljana（卢布尔雅那大学）

机构由 AI 辅助整理，请以论文原文为准。