级联辩论:面向成本感知的LLM级联的多智能体辩论
CascadeDebate: Multi-Agent Deliberation for Cost-Aware LLM Cascades
AI总结:
本文提出CascadeDebate,通过在每个层级的升级边界插入多智能体辩论,解决单模型层级在模糊查询中易引发升级的问题,提升准确性和成本效率。
摘要:
级联LLM系统通过协调不同规模的模型与人类专家,在不确定性下平衡准确性、成本和回避。然而,单模型层级在处理模糊查询时往往因不自信和计算效率低下而提前升级到更昂贵的模型或专家。CascadeDebate通过在每个层级的升级边界插入多智能体辩论来解决这一问题。基于置信度的路由器仅在不确定情况下激活轻量级智能体集合,使内部达成共识解决模糊性,而无需调用更高成本的升级。我们的统一架构在不同模型规模间交替单模型推理与选择性多智能体辩论,最终以人类专家作为最终后备。该设计根据查询难度动态扩展测试时间计算。在涵盖科学、医学和一般知识的五个基准测试中,CascadeDebate在强单模型级联和独立多智能体系统上表现提升最高达26.75%。一个在线阈值优化器证明至关重要,相对于固定策略,其准确性提升20.98至52.33个百分点,能够实现弹性适应现实世界分布。
英文摘要:
Cascaded LLM systems coordinate models of varying sizes with human experts to balance accuracy, cost, and abstention under uncertainty. However, single-model tiers at each stage often struggle with ambiguous queries, triggering premature escalations to costlier models or experts due to under-confidence and inefficient compute scaling. CascadeDebate addresses this gap by inserting multi-agent deliberation directly at each tier's escalation boundary. Confidence-based routers activate lightweight agent ensembles only for uncertain cases, enabling consensus-driven resolution of ambiguities internally without invoking higher-cost upgrades. Our unified architecture alternates single-model inference with selective multi-agent deliberation across model scales, culminating in human experts as the final fallback. This design scales test-time compute dynamically according to query difficulty. Across five benchmarks spanning science, medicine, and general knowledge, CascadeDebate outperforms strong single-model cascades and standalone multi-agent systems by up to 26.75 percent. An online threshold optimizer proves essential, boosting accuracy by 20.98 to 52.33 percent relative improvement over fixed policies and enabling elastic adaptation to real-world distributions.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。