发表机构
Federal Rural University of Pernambuco; University of Naples Federico II; Gran Sasso Science Institute (GSSI)（伯南布哥联邦农村大学; 那不勒斯费德里科二世大学; 格兰萨索科学研究所）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文实验研究了LLM生成的多语言服务应用的软件老化，发现内存使用是最一致的老化指标，仅功能正确性不足以评估其持续运行的可靠性。
AI 中文摘要
大语言模型（LLM）越来越多地被用于从自然语言规范生成可执行软件系统，这加速了开发进程并减少了手动实现的工作量。尽管近期研究已探讨了LLM生成代码的功能正确性、安全性、可维护性和鲁棒性，但人们对这类系统在持续执行下的长期可靠性知之甚少。本文通过实验研究了不同编程语言下LLM生成的基于服务的应用中的软件老化症状。我们使用BaxBench衍生的后端场景，通过基于LLM的生成平台生成了针对JavaScript、Python和Rust的应用，用BaxBench衍生的测试对其进行验证，并让它们经受48小时的工作负载执行。我们监测了内存使用量、响应时间和吞吐量，并使用Mann-Kendall检验和Sen斜率估计器对其进行分析。我们还通过对生成的源代码进行静态分析，并与人工编写的相关后端场景实现进行探索性比较，补充了运行时评估。结果表明，内存使用量是潜在软件老化最一致的指标，在大多数应用-语言组合中呈现出统计上显著的上升趋势，而响应时间和吞吐量则表现出更异质的行为。静态分析确定了合理的代码级老化机制，与人工编写系统的比较显示，手动开发的实现中也可能出现老化趋势。这些发现表明，在持续运行环境中部署前，仅靠功能正确性不足以评估LLM生成软件的运行可靠性。
英文摘要
Large Language Models (LLMs) are increasingly used to generate executable software systems from natural language specifications, accelerating development and reducing manual implementation effort. Although recent studies have investigated the functional correctness, security, maintainability, and robustness of LLM-generated code, little is known about the long-term reliability of such systems under sustained execution. In this paper, we experimentally investigate software aging symptoms in LLM-generated service-based applications across generation-and-execution environments. Using backend scenarios derived from BaxBench, we generated applications targeting JavaScript, Python, and Rust through LLM-based generation platforms, validated them with BaxBench-derived tests, and subjected them to 48-hour workload executions. We monitored memory usage, response time, and throughput and analyzed them using the Mann--Kendall test and Sen's slope estimator. We further complemented the runtime evaluation with static analysis of the generated source code and an exploratory comparison with human-written implementations of related backend scenarios. The results show that memory usage is the most consistent indicator of potential software aging, with statistically significant upward trends in most application-environment combinations, while response time and throughput exhibit more heterogeneous behavior. Static analysis identified plausible code-level aging mechanisms, and the comparison with human-written systems showed that the aging symptoms observed in LLM-generated applications align with degradation patterns also found in manually developed implementations. These findings indicate that functional correctness alone is insufficient to assess the operational reliability of LLM-generated software before deployment in continuously running environments.
