MIRAGE-Bench：LLM 智能体正在产生幻觉以及在哪里发现它们
MIRAGE-Bench: LLM Agent is Hallucinating and Where to Find Them
AI总结：
本文提出 MIRAGE-Bench，通过三类幻觉分类、快照决策点测试用例和细粒度 LLM-as-a-Judge 评估，系统识别交互式 LLM 智能体的幻觉动作。
摘要：
幻觉给基于大型语言模型（LLM）的智能体带来了严重风险，通常表现为由认知情境中捏造或误解的信息所导致的幻觉性行为。尽管近期研究已经揭示了此类失败，但现有评估仍然分散，且缺乏有原则的测试平台。本文提出 MIRAGE-Bench——Measuring Illusions in Risky AGEnt settings（在风险智能体环境中测量错觉）——这是首个用于在交互式 LLM 智能体场景中诱发和评估幻觉的统一基准。我们首先引入一种三部分分类法来处理智能体幻觉：不忠实于（i）任务指令、（ii）执行历史或（iii）环境观察的动作。为进行分析，我们首先通过系统审计现有智能体基准来诱发此类失败，随后使用快照策略合成测试用例，该策略以确定性和可复现的方式隔离决策点。为评估幻觉行为，我们采用细粒度层级的 LLM-as-a-Judge 范式，并配有定制的风险感知提示，从而无需枚举完整动作空间即可对智能体动作进行可扩展、高保真的评估。MIRAGE-Bench 提供了关于 LLM 智能体失败模式的可行动洞见，并为在交互式环境中缓解幻觉的有原则进展奠定基础。
英文摘要：
Hallucinations pose critical risks for large language model (LLM)-based agents, often manifesting as hallucinative actions resulting from fabricated or misinterpreted information within the cognitive context. While recent studies have exposed such failures, existing evaluations remain fragmented and lack a principled testbed. In this paper, we present MIRAGE-Bench--Measuring Illusions in Risky AGEnt settings--the first unified benchmark for eliciting and evaluating hallucinations in interactive LLM-agent scenarios. We begin by introducing a three-part taxonomy to address agentic hallucinations: actions that are unfaithful to (i) task instructions, (ii) execution history, or (iii) environment observations. To analyze, we first elicit such failures by performing a systematic audit of existing agent benchmarks, then synthesize test cases using a snapshot strategy that isolates decision points in deterministic and reproducible manners. To evaluate hallucination behaviors, we adopt a fine-grained-level LLM-as-a-Judge paradigm with tailored risk-aware prompts, enabling scalable, high-fidelity assessment of agent actions without enumerating full action spaces. MIRAGE-Bench provides actionable insights on failure modes of LLM agents and lays the groundwork for principled progress in mitigating hallucinations in interactive environments.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
