扩散语言模型在代理工作流中的苦涩教训:全面的现实检验
The Bitter Lesson of Diffusion Language Models for Agentic Workflows: A Comprehensive Reality Check
AI总结:
本文评估了扩散语言模型在代理任务中的表现,发现其在长期规划和工具调用任务中存在系统性失败,提出需整合因果推理机制以提升代理能力。
摘要:
实时代理交互的追求推动了扩散基于大语言模型(dLLMs)作为替代自回归架构的研究,旨在突破序列延迟瓶颈。然而,这种效率提升是否能转化为有效的代理行为?本文全面评估了dLLMs(如LLaDA、Dream)在两种代理范式中的表现:具身代理(需长期规划)和工具调用代理(需精确格式)。与效率炒作相反,我们的Agentboard和BFCL实验揭示了“苦涩教训”:当前dLLMs无法作为可靠的代理核心,频繁导致系统性失败。在具身场景中,dLLMs反复尝试,无法在时间反馈下分支;在工具调用场景中,dLLMs在扩散噪声下无法维持符号精度(如严格JSON模式)。为评估dLLMs在代理工作流中的潜力,我们引入DiffuAgent框架,整合dLLMs作为即插即用的认知核心。分析显示,dLLMs在非因果角色(如记忆摘要和工具选择)中有效,但需在去噪过程中整合因果、精确和逻辑基础的推理机制,才能适用于代理任务。
英文摘要:
The pursuit of real-time agentic interaction has driven interest in Diffusion-based Large Language Models (dLLMs) as alternatives to auto-regressive backbones, promising to break the sequential latency bottleneck. However, does such efficiency gains translate into effective agentic behavior? In this work, we present a comprehensive evaluation of dLLMs (e.g., LLaDA, Dream) across two distinct agentic paradigms: Embodied Agents (requiring long-horizon planning) and Tool-Calling Agents (requiring precise formatting). Contrary to the efficiency hype, our results on Agentboard and BFCL reveal a "bitter lesson": current dLLMs fail to serve as reliable agentic backbones, frequently leading to systematically failure. (1) In Embodied settings, dLLMs suffer repeated attempts, failing to branch under temporal feedback. (2) In Tool-Calling settings, dLLMs fail to maintain symbolic precision (e.g. strict JSON schemas) under diffusion noise. To assess the potential of dLLMs in agentic workflows, we introduce DiffuAgent, a multi-agent evaluation framework that integrates dLLMs as plug-and-play cognitive cores. Our analysis shows that dLLMs are effective in non-causal roles (e.g., memory summarization and tool selection) but require the incorporation of causal, precise, and logically grounded reasoning mechanisms into the denoising process to be viable for agentic tasks.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。