发表机构
University of Cambridge; Huawei; HKUST (GZ)（剑桥大学; 华为; 香港科技大学（广州））
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本研究提出可执行变换框架，系统测量LLM智能体在等价工具模式下的模式偏差，发现偏差显著且训练仅能修复训练数据中出现的变体。
AI 中文摘要
大型语言模型（LLMs）在给定固定工具模式时，在工具使用型智能体任务上表现出强大性能。然而，工具模式并非智能体的动作空间，它仅仅是动作空间的一种接口表示。同一可执行动作可以通过许多不同但功能等价（functionally equivalent）的工具定义来暴露，而真正学会任务的智能体应在这些不同定义下表现一致。我们表明，当前智能体往往并非如此，我们将这一现象称为模式偏差（schema bias）。为系统研究该现象，我们引入了一个可执行变换框架，该框架使用九种算子重写原生工具模式，包括合并和拆分工具、改变单个工具的表示方式，以及将一个动作分布到多个依赖调用中。任务、可执行动作和可达状态保持不变，因此任何成功率变化都可单独归因于接口本身。我们评估了十一款LLM（包括两款闭源模型），在多达32种模式变体上，探究模式偏差有多大、如何表现、是否无需完整评估即可预测某模式变体的难度，以及训练是否能消除该偏差。我们发现，即使对最新模型，模式偏差也相当显著：仅取决于模式，成功率从完全失败到97%不等。要可靠估计模式难度，需运行目标查询的小样本。训练仅当某模式变体出现在训练数据中时才能修复该变体。
英文摘要
Large Language Models (LLMs) have shown strong performance on tool-use agentic tasks when given a fixed tool schema. Yet a tool schema is not the action space of an agent; it is merely one interface representation of it. The same executable action can be exposed through many different, functionally equivalent tool definitions, and an agent that has truly learned a task should behave consistently across them. We show that current agents often do not, a phenomenon we term schema bias. To study this systematically, we introduce an executable transformation framework that rewrites a native tool schema using nine operators, including merging and splitting tools, altering how a single tool is expressed, and distributing one action across several dependent calls. The tasks, executable actions, and reachable states remain fixed, so any change in success is attributable to the interface alone. Evaluating eleven LLMs, including two closed models, on up to 32 schema variants, we ask how large schema bias is, how it manifests, whether the difficulty of a schema variant can be predicted without a full evaluation, and whether training removes it. We find that schema bias is substantial even for the newest models: success rates range from complete failure to 97% depending solely on the schema. To reliably estimate schema difficulty, it requires running a small sample of the target queries. Training repairs a schema variant only when that variant appears in the training data.
