对现代LLM代理框架中缺陷的实证研究
Where Agent Frameworks Fall Short: Examining Functional Challenges and Usability Concerns
AI总结：
本文通过分析998个缺陷报告，发现现代LLM代理框架中缺陷主要源于API误用、不兼容和文档不同步，集中在自我行动阶段，导致功能错误、崩溃和构建失败。
摘要：
LLM代理已被广泛应用于现实世界的应用中，依赖于代理框架进行工作流执行和多代理协调。随着这些系统规模的扩大，理解底层代理框架中的缺陷变得至关重要。然而，现有研究主要关注代理层面的失败，忽略了框架层面的缺陷。为了解决这一缺口，我们对CrewAI和LangChain的998个缺陷报告进行了实证研究，构建了一个包含15个根本原因和7个可观察症状的分类学，涵盖五个代理生命周期阶段：'代理初始化'、'感知'、'自我行动'、'相互交互'和'进化'。我们的发现表明，代理框架中的缺陷主要源于'API误用'、'API不兼容'和'文档不同步'，主要集中在'自我行动'阶段。症状通常表现为'功能错误'、'崩溃'和'构建失败'，反映了对任务进展和控制流的破坏。
英文摘要：
Large language model (LLM) agents are increasingly built on agent frameworks that provide reusable abstractions for workflow orchestration, state management, tool integration, and execution control. However, the quality of this infrastructure layer remains insufficiently understood, particularly its functionality challenges and usability concerns, as existing studies have mainly examined traditional deep learning (DL) frameworks or model-level agent failures. Therefore, we conduct an empirical study of 5,669 bug reports and 809 feature requests from five mainstream agent frameworks: AutoGen, CrewAI, LangChain, LangGraph, and MetaGPT. We construct a four-dimensional taxonomy covering 22 root causes, seven symptoms, 11 motivations, and six requirements, and map them to the five-stage agent lifecycle. Across the four RQs, results show an execution centered quality pattern shaped by semantic interface boundaries. Reported bugs mainly manifest as Incorrect Functionality (76.00%) and involve more API, configuration, parsing, and serialization related causes than DL framework bugs, while their associations remain sparse and stage specific. Feature requests mainly target Feature Enhancement (49.07%) and reveal structured needs for Orchestration Expressiveness, Development Delivery, Model Adaptation, and Tool Ecosystem. These findings call for quality assurance beyond crash based and tensor level testing, with emphasis on API sequences, structured LLM outputs, serialization boundaries, execution traces, and execution centered maintenance, offering empirical guidance for reliable and usable agent framework infrastructure.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
