发表机构
Leiden University(莱顿大学)
机构由 AI 辅助整理,请以论文原文为准。AI 中文总结
本研究通过配对实验对比扁平与层级LLM智能体团队,发现层级权威的修订循环降低输出质量,监督者仅在可验证时才有价值。
AI 中文摘要
层级编排,即管理者智能体审查工人输出并可将其退回修改,是生产级多智能体LLM框架中的默认协调模式。经典组织理论预测,权威链接能加速达成决定性输出;而关于奉承行为和思维退化(Degeneration-of-Thought)的研究则预测,权威性批评会使LLM输出质量变差。以往的比较研究在具有可核查答案的任务上整体更换框架,未能在开放式工作中单独测试权威链接。我们提出一项配对实验,固定五个LLM智能体及其角色、提示词、工具、模型和数据不变,仅改变一个链接:管理者是否可以拒绝工人的输出并要求修改。在43个配对产品和86次商业智能报告任务运行中,一个由五个模型组成的评审小组和一项确定性规范检查对每份报告进行评分。扁平组织在实用性(d = 0.42, p = 0.009)和写作清晰度(d = 0.34, p = 0.030)上得分更高;经典预测失败。报告长度相同,但层级报告对冲表述多出53%,每次修订循环与写作清晰度下降0.14分相关,且层级制写作者的首稿与扁平报告无法区分:差距在修订循环内部显现。两种组织的规范准确性均达到上限,监督层级额外消耗51.5%的token却未带来质量提升。监督者只有在能验证时才有价值,当只能发表意见时则成为负担。
英文摘要
Does authority in AI teams improve the outcome? Organizational theory asserts that authority facilitates decision making, improving quality. Meanwhile, some nascent AI research suggests that revision under authority makes LLM output worse. Multi-agent LLM frameworks default to giving a Manager agent the authority to send a worker's output back for revision. Prior comparisons test the effect of authority using verifiable tasks. We conduct an experiment on an open-ended task, business-intelligence reporting, using a sample of 43 paired laptop products and 86 runs. Each report is written once by a hierarchical team and once by a flat team. We find that flat teams produce higher-quality reports, scoring higher on Utility (d = 0.42, p = 0.009) and Writing Clarity (d = 0.34, p = 0.030). The reports are the same length, but hierarchical team reports use 53% more hedging words such as "may" and "could", and each revision is associated with a 0.14-point drop in Writing Clarity on a 1 to 5 scale. Before any revision, the hierarchical team's first draft is indistinguishable from the flat team's report. In other words, the quality gap can be traced to revision. Authority improves quality when the Manager can verify the work, else when it can only provide feedback it has a negative effect on quality.
Comments8 pages, 3 figures, 3 tables, plus 21 pages of supplementary material. Code: https://github.com/cihatburak/loop-back-authority-llm-agents