arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2408.13406cs.AIcs.CEcs.MA
收藏

有限元分析中多智能体大语言模型系统的协作动力学与可靠性挑战

Collaboration Dynamics and Reliability Challenges of Multi-Agent LLM Systems in Finite Element Analysis

University of Canterbury(坎特伯雷大学)

Chuan Tian, Yilei Zhang

AI总结:

本文通过 AutoGen 多智能体框架在有限元分析中的 1,120 次试验,揭示协作效果取决于功能互补而非团队规模,并针对确认偏差、过早共识和验证-确认差距三类失效模式提出互补角色、多级验证和对抗性交互控制等设计原则。

摘要:

基于大语言模型(LLM)的多智能体系统正越来越多地应用于自动化科学与工程中的计算工作流。然而,智能体之间的动态如何影响推理质量和验证可靠性仍不清楚。我们使用一个基于 AutoGen 的多智能体框架研究这些机制,该框架用于线弹性有限元分析(FEA),在固定的 12 轮对话限制下,跨四个任务评估七种角色配置。从 1,120 次受控试验中,我们发现协作有效性更多地取决于功能互补性而非团队规模:三智能体的 Coder-Executor-Critic 配置唯一地产生了物理上和视觉上正确的解,而添加冗余的审查者则降低了成功率。然而,三种系统性失效模式仍然存在:(1)确认偏差,即 Rebuttal 智能体认可而非质疑输出(85-92% 的一致率,包括错误);(2)由冗余审查者引起的过早共识;(3)验证-确认差距,即可执行但物理上不正确的代码未被检测到地通过。没有任何智能体组合能在复杂任务中成功验证本构关系。基于功能多样性、角色分化和计算验证理论,我们提出了可操作的设计原则:(i)分配互补的智能体角色,(ii)强制执行多级验证(执行、规范、物理),以及(iii)通过对抗性或基于触发器的交互控制防止过早共识。这些发现为在工程工作流中设计可信的 LLM 协作奠定了原则性基础。

英文摘要:

Large Language Model (LLM)-based multi-agent systems are increasingly applied to automate computational workflows in science and engineering. However, how inter-agent dynamics influence reasoning quality and verification reliability remains unclear. We study these mechanisms using an AutoGen-based multi-agent framework for linear-elastic Finite Element Analysis (FEA), evaluating seven role configurations across four tasks under a fixed 12-turn conversation limit. From 1,120 controlled trials, we find that collaboration effectiveness depends more on functional complementarity than team size: the three-agent Coder-Executor-Critic configuration uniquely produced physically and visually correct solutions, while adding redundant reviewers reduced success rates. Yet three systematic failure modes persist: (1) affirmation bias, where the Rebuttal agent endorsed rather than challenged outputs (85-92% agreement, including errors); (2) premature consensus caused by redundant reviewers; and (3) a verification-validation gap where executable but physically incorrect code passed undetected. No agent combination successfully validated constitutive relations in complex tasks. Building on theories of functional diversity, role differentiation, and computational validation, we propose actionable design principles: (i) assign complementary agent roles, (ii) enforce multi-level validation (execution, specification, physics), and (iii) prevent early consensus through adversarial or trigger-based interaction control. These findings establish a principled foundation for designing trustworthy LLM collaborations in engineering workflows.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑