MMAC-Copilot：多模态智能体协作操作副驾驶
MMAC-Copilot: Multi-modal Agent Collaboration Operating Copilot
AI总结：
针对大语言模型智能体与PC交互模式单一、通用性差且易产生幻觉的问题，提出多模态智能体协作框架MMAC-Copilot，通过团队协作链整合多智能体领域知识，在GAIA基准上较现有系统提升6.8%，还构建了VIBench基准验证其在非API应用中的能力。
摘要：
与PC应用交互的大语言模型智能体常因与现实环境交互模式单一而存在局限，导致通用性受限且频繁出现幻觉。为解决该问题，我们提出多模态智能体协作框架（MMAC-Copilot），该框架利用不同智能体的集体专业知识增强与应用的交互能力。框架引入团队协作链，使每个参与智能体基于自身特定领域知识贡献见解，有效减少因知识领域缺口带来的幻觉。我们使用GAIA基准和我们新提出的视觉交互基准（VIBench）评估MMAC-Copilot。MMAC-Copilot在GAIA上取得了优异性能，相比现有领先系统平均提升6.8%。VIBench聚焦各领域中无法通过API交互的应用，涵盖3D游戏、娱乐和办公场景。MMAC-Copilot在VIBench上也展现出了出色能力。我们希望本工作能为该领域带来启发，并为自主智能体提供更全面的评估。匿名Github地址为：https://anonymous.4open.science/r/ComputerAgentWithVision-3C12
英文摘要：
Large language model agents that interact with PC applications often face limitations due to their singular mode of interaction with real-world environments, leading to restricted versatility and frequent hallucinations. To address this, we propose the Multi-Modal Agent Collaboration framework (MMAC-Copilot), a framework utilizes the collective expertise of diverse agents to enhance interaction ability with application. The framework introduces a team collaboration chain, enabling each participating agent to contribute insights based on their specific domain knowledge, effectively reducing the hallucination associated with knowledge domain gaps. We evaluate MMAC-Copilot using the GAIA benchmark and our newly introduced Visual Interaction Benchmark (VIBench). MMAC-Copilot achieved exceptional performance on GAIA, with an average improvement of 6.8\% over existing leading systems. VIBench focuses on non-API-interactable applications across various domains, including 3D gaming, recreation, and office scenarios. It also demonstrated remarkable capability on VIBench. We hope this work can inspire in this field and provide a more comprehensive assessment of Autonomous agents. The anonymous Github is available at \href{https://anonymous.4open.science/r/ComputerAgentWithVision-3C12}{Anonymous Github}
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
