HerAgent: 通过分层测试金字塔重新思考自动化环境部署
HerAgent: Rethinking the Automated Environment Deployment via Hierarchical Test Pyramid
AI总结:
HerAgent通过分层测试金字塔方法,利用执行验证和修复实现自动化环境部署,显著提升环境配置成功率。
摘要:
自动化软件环境设置是测试、调试和重现故障的前提,但在实践中由于复杂的依赖关系、异构的构建系统和不完整的文档而仍然具有挑战性。最近的工作利用大语言模型来自动化这一过程,但通常使用弱信号如依赖安装或部分测试执行来评估成功,这些并不能确保项目实际上可以运行。在本文中,我们主张通过可执行证据来评估环境设置的成功,而不是单一的二进制信号。我们引入了环境成熟度层次,该层次基于逐步增强的执行要求定义了三个成功级别,最终达到成功执行项目的主要入口点。基于这一层次,我们提出了HerAgent,一种自动化环境设置方法,通过基于执行的验证和修复逐步构建可执行环境。我们在四个公开基准上评估了HerAgent,其在相关工作中表现优异,由于其对项目结构和依赖的全面理解,实现了高达79.6%的改进。在复杂的C/C++项目中,HerAgent比先前的方法高出66.7%。此外,HerAgent在基准上唯一解决了11-30个环境实例,这些实例无法由先前的方法配置。
英文摘要:
Automated software environment setup is a prerequisite for testing, debugging, and reproducing failures, yet remains challenging in practice due to complex dependencies, heterogeneous build systems, and incomplete documentation. Recent work leverages large language models to automate this process, but typically evaluates success using weak signals such as dependency installation or partial test execution, which do not ensure that a project can actually run. In this paper, we argue that environment setup success should be evaluated through executable evidence rather than a single binary signal. We introduce the Environment Maturity Hierarchy, which defines three success levels based on progressively stronger execution requirements, culminating in successful execution of a project's main entry point. Guided by this hierarchy, we propose HerAgent, an automated environment setup approach that incrementally constructs executable environments through execution-based validation and repair. We evaluate HerAgent on four public benchmarks, where it outperforms all related work, achieving up to 79.6\% improvement due to its holistic understanding of project structure and dependencies. On complex C/C++ projects, HerAgent surpasses prior approaches by 66.7\%. In addition, HerAgent uniquely resolves 11-30 environment instances across the benchmarks that no prior method can configure.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。