Error attribution in Large Language Model (LLM) multi-agent systems presents a significant challenge in debugging and improving collaborative AI systems. Current approaches to pinpointing agent and step level failures in interaction traces - whether using all-at-once evaluation, step-by-step analysis, or binary search - fall short when analyzing complex patterns, struggling with both accuracy and consistency. We present ECHO (Error attribution through Contextual Hierarchy and Objective consensus analysis), a novel algorithm that combines hierarchical context representation, objective analysis-based evaluation, and consensus voting to improve error attribution accuracy. Our approach leverages a positional-based leveling of contextual understanding while maintaining objective evaluation criteria, ultimately reaching conclusions through a consensus mechanism. Experimental results demonstrate that ECHO outperforms existing methods across various multi-agent interaction scenarios, showing particular strength in cases involving subtle reasoning errors and complex interdependencies. Our findings suggest that leveraging these concepts of structured, hierarchical context representation combined with consensus-based objective decision-making, provides a more robust framework for error attribution in multi-agent systems.
Where Did It Go Wrong? Capability-Oriented Failure Attribution for Vision-and-Language Navigation Agents
Institute of Software, Chinese Academy of Sciences(中国科学院软件研究所);Science & Technology on Integrated Information System Laboratory(信息集成信息系统实验室);State Key Laboratory of Complex System Modeling and Simulation Technology(复杂系统建模与仿真技术国家重点实验室);University of Chinese Academy of Sciences(中国科学院大学);Singapore Management University(新加坡管理大学)
HANSARD: A Reference Architecture for Forensic Readiness, Runtime Witnessing, and Graded Attribution in Autonomous Multi-Agent AI Systems
Harokopio University of Athens(雅典哈罗科皮奥大学);Democritus University of Thrace(色雷斯德谟克利特大学);University of Western Macedonia(西马其顿大学);Athena Research Center(雅典娜研究中心)
University of Tübingen, Carl Friedrich von Weizsäcker Center(图宾根大学,卡尔·弗里德里希·冯·魏茨泽克中心);University of Geneva, Department of Philosophy(日内瓦大学哲学系);University of Zurich, Department of Philosophy(苏黎世大学哲学系)
Did Alice Do Wrong? Cross-Cultural Differences in Student Perceptions of Generative AI Use in University Computing Education
University of Toronto Scarborough(多伦多大学士嘉堡分校);Botswana International University of Science and Technology(博茨瓦纳国际科技大学);Sungkyunkwan University(成均馆大学);Open Polytechnic of New Zealand(新西兰开放理工学院)
ProvenanceGuard: Source-Aware Factuality Verification for MCP-Based LLM Agents
Multiverse Computing;Parque Cientifico y Tecnológico de Gipuzkoa(吉普斯夸科技园);Centre for Social Innovation(社会创新中心);Donostia International Physics Center(多诺斯蒂亚国际物理中心);Ikerbasque Foundation for Science(伊克尔巴斯克科学基金会)