Harness Engineering for Agentic AI Coding Tools: An Exploratory Study
University of Bamberg(班贝格大学);Heidelberg University(海德堡大学);Singapore Management University(新加坡管理大学)
Matthias Galster, Seyedmoein Mohsenimofidi, Jai Lal Lulla, Muhammad Auwal Abubakar, Christoph Treude, Sebastian Baltes
备注:
10 pages, 7 figures, 3 tables, based on "Configuring Agentic AI Coding Tools: An Exploratory Study" published in Proceedings of the 3rd ACM/IEEE International Conference on AI-powered Software (AIware 2026)
Agentic AI coding tools increasingly automate software development tasks. Developers can configure these tools through versioned repository-level artifacts such as Markdown and JSON files. We present a systematic analysis of configuration mechanisms for agentic AI coding tools, covering Claude Code, GitHub Copilot, Cursor, Gemini, and Codex. We identify eight configuration mechanisms spanning from static context to executable and external integrations and, in an empirical study of 2,853 GitHub repositories, examine whether and how they are adopted, with a detailed analysis of Context Files, Skills, and Subagents. First, Context Files dominate the configuration landscape and are often the sole mechanism in a repository, with AGENTS$.$md emerging as an interoperable standard across tools. Second, few repositories adopt advanced mechanisms such as Skills and Subagents. Skills predominantly rely on static instructions rather than executable scripts. Third, distinct configuration practices are forming around different tools, with Claude Code users employing the broadest range of mechanisms. These findings establish an empirical baseline for understanding how developers configure agentic tools, suggest that AGENTS$.$md serves as a natural starting point, and motivate longitudinal and experimental research on how configuration strategies evolve and affect agent performance.
AgentBug-Smith: Automatically Reproducing Real-World Harness Bugs in Agentic Systems
The University of Chicago(芝加哥大学);Fudan University(复旦大学);TensorBlock, Inc.(TensorBlock公司);Tsinghua University(清华大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校)