ProxyWar: 动态评估LLM代码生成在游戏竞技场中的表现
ProxyWar: Dynamic Assessment of LLM Code Generation in Game Arenas
AI总结：
ProxyWar通过在动态游戏环境中评估LLM代码生成的质量，揭示了基准分数与实际性能之间的差异，为算法发现和自适应问题解决研究提供新视角。
摘要：
大型语言模型（LLMs）已革新了自动化代码生成，但其在现实中的有效性评估仍受限于静态基准和简单的指标。我们提出了ProxyWar，一种新的框架，通过将LLM生成的代理嵌入到多样且竞争性的游戏环境中，系统地评估代码生成质量。与现有方法不同，ProxyWar不仅评估功能正确性，还评估生成程序的操作特性，结合自动化测试、迭代代码修复和多代理比赛，提供程序行为的全面视角。应用于一系列最先进的编码器和游戏后，我们的方法揭示了基准分数与动态设置中实际性能之间显著的差异，揭示了被忽视的限制和改进机会。这些发现突显了需要更丰富的、基于竞争的代码生成评估的必要性。展望未来，ProxyWar为研究LLM驱动的算法发现、自适应问题解决以及实际效率和鲁棒性的研究奠定了基础，包括模型可能超越手工制作代理的潜力。该项目可在https://github.com/xinke-wang/ProxyWar上获取。
英文摘要：
Large language models (LLMs) have revolutionized automated code generation, yet the evaluation of their real-world effectiveness remains limited by static benchmarks and simplistic metrics. We present ProxyWar, a novel framework that systematically assesses code generation quality by embedding LLM-generated agents within diverse, competitive game environments. Unlike existing approaches, ProxyWar evaluates not only functional correctness but also the operational characteristics of generated programs, combining automated testing, iterative code repair, and multi-agent tournaments to provide a holistic view of program behavior. Applied to a range of state-of-the-art coders and games, our approach uncovers notable discrepancies between benchmark scores and actual performance in dynamic settings, revealing overlooked limitations and opportunities for improvement. These findings highlight the need for richer, competition-based evaluation of code generation. Looking forward, ProxyWar lays a foundation for research into LLM-driven algorithm discovery, adaptive problem solving, and the study of practical efficiency and robustness, including the potential for models to outperform hand-crafted agents. The project is available at https://github.com/xinke-wang/ProxyWar.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
