真理是普适的：大语言模型中谎言的鲁棒检测
Truth is Universal: Robust Detection of Lies in LLMs
AI总结：
本文发现真伪陈述的激活向量可在普适的二维子空间中被分离，并据此构建了在多个LLM上达到94%准确率的鲁棒谎言检测器。
摘要：
大语言模型（LLMs）已经彻底改变了自然语言处理，展现出令人印象深刻的类人能力。特别是，LLMs能够“说谎”，即故意输出虚假陈述。因此，开发检测LLMs何时说谎的方法具有重要的研究意义和实用价值。事实上，已有若干研究者基于模型内部激活训练分类器来检测LLM谎言。然而，其他研究人员表明，这些分类器可能无法泛化，例如在遇到否定陈述时。在这项工作中，我们旨在开发一种鲁棒的方法来检测LLM何时在说谎。为此，我们做出了以下关键贡献：（i）我们证明存在一个二维子空间，沿着该子空间，真陈述和假陈述的激活向量可以被分离。值得注意的是，这一发现具有普适性，适用于多种LLMs，包括Gemma-7B、LLaMA2-13B、Mistral-7B和LLaMA3-8B。我们的分析解释了先前研究中观察到的泛化失败现象，并为更鲁棒的谎言检测奠定了基础；（ii）在（i）的基础上，我们构建了一个精确的LLM谎言检测器。实验表明，我们提出的分类器达到了最先进的性能，在区分真假事实陈述和检测真实世界场景中生成的谎言两个任务上均取得了94%的准确率。
英文摘要：
Large Language Models (LLMs) have revolutionised natural language processing, exhibiting impressive human-like capabilities. In particular, LLMs are capable of "lying", knowingly outputting false statements. Hence, it is of interest and importance to develop methods to detect when LLMs lie. Indeed, several authors trained classifiers to detect LLM lies based on their internal model activations. However, other researchers showed that these classifiers may fail to generalise, for example to negated statements. In this work, we aim to develop a robust method to detect when an LLM is lying. To this end, we make the following key contributions: (i) We demonstrate the existence of a two-dimensional subspace, along which the activation vectors of true and false statements can be separated. Notably, this finding is universal and holds for various LLMs, including Gemma-7B, LLaMA2-13B, Mistral-7B and LLaMA3-8B. Our analysis explains the generalisation failures observed in previous studies and sets the stage for more robust lie detection; (ii) Building upon (i), we construct an accurate LLM lie detector. Empirically, our proposed classifier achieves state-of-the-art performance, attaining 94% accuracy in both distinguishing true from false factual statements and detecting lies generated in real-world scenarios.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
