InEx：通过内省与跨模态多智能体协作缓解幻觉
InEx: Hallucination Mitigation via Introspection and Cross-Modal Multi-Agent Collaboration
AI总结：
InEx通过内省推理和跨模态多智能体协作，自主缓解大型语言模型的幻觉问题，实验表明其在多个基准上表现优异。
摘要：
幻觉仍然是大型语言模型（LLMs）中的关键挑战，阻碍了可靠多模态LLMs（MLLMs）的发展。现有解决方案往往依赖人工干预或未充分利用智能体自主缓解幻觉的能力。为了解决这些限制，我们受到人类在现实世界中做出可靠决策方式的启发。他们首先通过内省推理来减少不确定性并形成初步判断，然后依靠来自不同视角的外部验证来达成最终决策。受此认知范式启发，我们提出了InEx，一种无需训练的多智能体框架，旨在自主缓解幻觉。InEx引入了由基于熵的不确定性估计引导的内部内省推理，以提高决策智能体推理过程的可靠性。智能体首先生成响应，然后通过与编辑智能体和自我反思智能体的外部跨模态多智能体协作进行迭代验证和优化，进一步提高可靠性并缓解幻觉。大量实验表明，InEx在一般性和幻觉基准上均优于现有方法，实现了4%-27%的提升，并表现出强大的鲁棒性。
英文摘要：
Hallucination remains a critical challenge in large language models (LLMs), hindering the development of reliable multimodal LLMs (MLLMs). Existing solutions often rely on human intervention or underutilize the agent's ability to autonomously mitigate hallucination. To address these limitations, we draw inspiration from how humans make reliable decisions in the real world. They begin with introspective reasoning to reduce uncertainty and form an initial judgment, then rely on external verification from diverse perspectives to reach a final decision. Motivated by this cognitive paradigm, we propose InEx, a training-free, multi-agent framework designed to autonomously mitigate hallucination. InEx introduces internal introspective reasoning, guided by entropy-based uncertainty estimation, to improve the reliability of the decision agent's reasoning process. The agent first generates a response, which is then iteratively verified and refined through external cross-modal multi-agent collaboration with the editing agent and self-reflection agents, further enhancing reliability and mitigating hallucination. Extensive experiments show that InEx consistently outperforms existing methods, achieving 4%-27% gains on general and hallucination benchmarks, and demonstrating strong robustness.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
