迈向减轻基于大语言模型的智能体幻觉：渐进泛化边界探索与看门狗监测
Towards Mitigation of Hallucination for LLM-empowered Agents: Progressive Generalization Bound Exploration and Watchdog Monitor
AI总结：
研究针对大语言模型赋能智能体的幻觉问题，提出HalMit黑箱框架，用概率分形采样技术并行生成查询触发异常响应，以识别泛化边界来检测幻觉，提升智能体可靠性。
摘要：
由大语言模型赋能的智能体成为与开放环境交互的流行范式，但模型产生的幻觉对智能体可信度构成重大挑战。相关方法存在局限，为此提出HalMit这一新颖的黑箱看门狗框架，通过概率分形采样技术有效识别目标智能体的泛化边界，实验表明其在幻觉监测方面显著优于现有方法。
英文摘要：
Empowered by large language models (LLMs), intelligent agents have become a popular paradigm for interacting with open environments to facilitate AI deployment. However, hallucinations generated by LLMs-where outputs are inconsistent with facts-pose a significant challenge, undermining the credibility of intelligent agents. Only if hallucinations can be mitigated, the intelligent agents can be used in real-world without any catastrophic risk. Therefore, effective detection and mitigation of hallucinations are crucial to ensure the dependability of agents. Unfortunately, the related approaches either depend on white-box access to LLMs or fail to accurately identify hallucinations. To address the challenge posed by hallucinations of intelligent agents, we present HalMit, a novel black-box watchdog framework that models the generalization bound of LLM-empowered agents and thus detect hallucinations without requiring internal knowledge of the LLM's architecture. Specifically, a probabilistic fractal sampling technique is proposed to generate a sufficient number of queries to trigger the incredible responses in parallel, efficiently identifying the generalization bound of the target agent. Experimental evaluations demonstrate that HalMit significantly outperforms existing approaches in hallucination monitoring. Its black-box nature and superior performance make HalMit a promising solution for enhancing the dependability of LLM-powered systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
