人工智能编码工具中的工程陷阱：对Claude代码、Codex和Gemini CLI中bug的实证研究
Engineering Pitfalls in AI Coding Tools: An Empirical Study of Bugs in Claude Code, Codex, and Gemini CLI
AI总结：
本文通过分析超过3800个公开报告的bug，揭示了构建AI编码工具中的常见故障模式和工程挑战，发现功能相关bug占比超67%，主要症状包括API错误、终端问题和命令失败。
摘要：
大型语言模型（LLMs）快速融入软件开发流程，催生了如Claude-Code、Codex和Gemini CLI等AI辅助编码工具。尽管有望提升生产力，但这些工具在传统软件工程、AI系统设计和人机交互的复杂交汇处构建过程中存在独特的工程挑战。本文首次对这些工具的工程陷阱进行了实证研究，通过对三个AI辅助编码工具的开源仓库中超过3800个公开报告的bug进行系统性手动分析。我们采用开放编码方法，手动检查问题描述、相关用户讨论和开发者回应。通过此过程，我们将每个bug按多个维度分类，包括bug类型、bug位置、根本原因和观察到的症状。这种细粒度标注使我们能够描述常见的故障模式并识别反复出现的工程挑战。我们的结果表明，这些工具中的bug超过67%与功能相关。在根本原因方面，36.9%的bug源于API、集成或配置错误。因此，用户最常报告的症状是API错误（18.3%）、终端问题（14%）和命令失败（12.7%）。这些bug主要影响系统工作流程中的工具调用（37.2%）和命令执行（24.7%）阶段。总体而言，我们的发现为开发者设计下一代可靠且稳健的AI编码助手提供了关键路线图。
英文摘要：
The rapid integration of Large Language Models (LLMs) into software development workflows has given rise to a new class of AI-assisted coding tools, such as Claude-Code, Codex, and Gemini CLIs. While promising significant productivity gains, the engineering process of building these tools, which sit at the complex intersection of traditional software engineering, AI system design, and human-computer interaction, is fraught with unique and poorly understood challenges. This paper presents the first empirical study of engineering pitfalls in building such tools, on a systematic, manual analysis of over 3.8K publicly reported bugs in the open-source repositories of three AI-assisted coding tools (i.e., Claude-Code, Codex, and Gemini CLIs) on GitHub. Specifically, we employ an open-coding methodology to manually examine the issue description, associated user discussions, and developer responses. Through this process, we categorize each bug along multiple dimensions, including bug type, bug location, root cause, and observed symptoms. This fine-grained annotation enables us to characterize common failure patterns and identify recurring engineering challenges. Our results show that more than 67% of the bugs in these tools are related to functionality. In terms of root causes, 36.9% of the bugs stem from API, integration, or configuration errors. Consequently, the most commonly observed symptoms reported by users are API errors (18.3%), terminal problems (14%), and command failures (12.7%). These bugs predominantly affect the tool invocation (37.2%) and command execution (24.7%) stages of the system workflow. Collectively, our findings provide a critical roadmap for developers seeking to design the next generation of reliable and robust AI coding assistants.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
