希特勒相关场景该如何处理？HAUNT：一种通过对抗性诱导探测大语言模型自一致性的框架
What About the Scene with the Hitler Reference? HAUNT: A Framework to Probe LLMs' Self-consistency Via Adversarial Nudge
AI总结：
针对大语言模型的幻觉问题，本文提出HAUNT框架，通过三步对抗性诱导测试模型自一致性，评估显示不同模型韧性差异显著，为高风险场景LLM应用的事实保真度提供警示。
摘要：
幻觉问题对大语言模型（LLMs）在高风险领域的实际部署构成重大挑战。本文提出一种在对抗性诱导存在时对LLMs事实保真度进行压力测试的框架。该框架包含三个步骤：第一步，指示LLM生成与特定封闭域一致的事实和谎言集合；第二步，指示LLM验证同一组断言是否为该封闭域下的事实或谎言；第三步，测试LLM对自身生成（并验证）的谎言的鲁棒性。我们使用五个知名专有LLM在热门电影和小说两个封闭域开展的广泛评估显示，模型对对抗性诱导的易感性差异显著：Claude表现出强韧性，GPT和Grok表现出中等韧性，而Gemini和DeepSeek表现出弱韧性。考虑到越来越多的人使用LLMs获取信息，我们的发现敲响了警钟。
英文摘要：
Hallucinations pose a critical challenge to the real-world deployment of large language models (LLMs) in high-stakes domains. In this paper, we present a framework for stress testing factual fidelity in LLMs in the presence of adversarial nudge. Our framework consists of three steps. In the first step, we instruct the LLM to produce sets of truths and lies consistent with the closed domain in question. In the next step, we instruct the LLM to verify the same set of assertions as truths and lies consistent with the same closed domain. In the final step, we test the robustness of the LLM against the lies generated (and verified) by itself. Our extensive evaluation, conducted using five widely known proprietary LLMs across two closed domains of popular movies and novels, reveals a wide range of susceptibility to adversarial nudges: \texttt{Claude} exhibits strong resilience, \texttt{GPT} and \texttt{Grok} demonstrate moderate resilience, while \texttt{Gemini} and \texttt{DeepSeek} show weak resilience. Considering that a large population is increasingly using LLMs for information seeking, our findings raise alarm.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
