arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2605.23916cs.IRcs.AIecon.GNq-fin.EC
收藏

面向智能体的LLM工具注册表信息设计

Advertising to Agents: Measuring Description-Framing Bias in LLM Tool Selection

Massachusetts Institute of Technology(麻省理工学院)

Haochuan Kevin Wang

AI总结:

本研究首次系统性地分析了LLM工具注册表中广告式描述对智能体选择的影响,发现法律上允许的夸大宣传(如主观最高级表述)完全主导优化效果,而虚假声明无额外影响,并提出了分离选择导向与营销导向描述及智能体注意力质量分数等注册表设计建议。

摘要:

LLM工具注册表作为未受监管的广告平台运作:提供者编写自由文本描述,智能体据此进行选择,但缺乏衡量基础设施——无可见性标准、质量评分或结果审计——来使该市场承担责任。我们提供了首个系统性框架,结合了跨越五个LLM和十个领域的17,700多次试验以及建设性的注册表设计处方。仅法律上的夸大宣传(主观最高级表述、利益框架)就捕获了100%的优化效果;虚假声明未增加任何额外偏差——这使得FTC对欺骗性广告规则的执法对活跃机制无效。信息披露在结构上失败:系统提示警告对五个模型中的四个产生零可测量效果,行为上限使得基于标签的修正没有空间。最高级表述是主导单一特征(SBC = +0.35)。注册表层的描述规范化实现了与模型无关的一流福利。我们提出将面向选择的描述(结构化的、注册表控制的)与面向营销的描述(提供者撰写的、选择后展示)分离,并引入智能体注意力质量分数以区分能力与文案撰写。

英文摘要:

AI agents often pick tools from registries, where each provider writes its tool's description; we ask whether sales language there changes which tool an agent picks. We built pairs of listings that differ in one way (praise, a verifiable specification, or list order) and asked two OpenAI models to call one tool. In a preregistered study, stacked praise (four kinds) raised a tool's pick rate by about 43 percentage points, matching or beating a verifiable specification. Praise also pulled some picks toward tools whose numeric limit was too small. With identical listings, the first-listed tool was picked about 72 points more often. Structured fields helped agents pick the capable tool; sales text beside them reduced or erased that gain. An exploratory study of five models shows the same direction; a "Sponsored" label dampened it, and a system-prompt warning moved one model. Registries could list limits as fields, hide sales text from agents, and randomize order. Stacked praise replicated on held-out domains; no single kind did after correcting for multiple tests. Preregistered results are provisional until blind phrase ratings are complete, and cover two small models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑