关于大语言模型内部通用真实性超平面的研究
On the Universal Truthfulness Hyperplane Inside LLMs
AI总结：
本文通过扩大训练数据集多样性并评估跨任务、跨领域及领域内泛化，发现大语言模型内部可能存在区分事实正确与错误输出的通用真实性超平面，且数据多样性比数据量更关键。
摘要：
尽管大语言模型（LLMs）在各个领域展现出卓越的能力，幻觉仍然是一个重大挑战。近期研究通过内部表征的视角探索幻觉现象，提出了用于解读LLMs对事实遵循程度的机制。然而，这些方法通常无法泛化到分布外数据，从而引发担忧：内部表征模式究竟反映的是根本性的事实意识，还是仅仅在特定数据集上过拟合了虚假相关性。在本工作中，我们研究模型内部是否存在一个通用真实性超平面，用以区分模型事实正确与事实错误的输出。为此，我们扩大了训练数据集的数量并进行了广泛评估——我们在超过40个数据集的多样化集合上训练真实性超平面，并检验其跨任务、跨领域和领域内泛化能力。我们的结果表明，提高训练数据集的多样性能够显著提升所有场景下的性能，而数据样本的数量则发挥相对次要的作用。这一发现支持了一个乐观的假设，即模型内部可能确实存在一个通用真实性超平面，为未来研究提供了有前景的方向。
英文摘要：
While large language models (LLMs) have demonstrated remarkable abilities across various fields, hallucination remains a significant challenge. Recent studies have explored hallucinations through the lens of internal representations, proposing mechanisms to decipher LLMs' adherence to facts. However, these approaches often fail to generalize to out-of-distribution data, leading to concerns about whether internal representation patterns reflect fundamental factual awareness, or only overfit spurious correlations on the specific datasets. In this work, we investigate whether a universal truthfulness hyperplane that distinguishes the model's factually correct and incorrect outputs exists within the model. To this end, we scale up the number of training datasets and conduct an extensive evaluation -- we train the truthfulness hyperplane on a diverse collection of over 40 datasets and examine its cross-task, cross-domain, and in-domain generalization. Our results indicate that increasing the diversity of the training datasets significantly enhances the performance in all scenarios, while the volume of data samples plays a less critical role. This finding supports the optimistic hypothesis that a universal truthfulness hyperplane may indeed exist within the model, offering promising directions for future research.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
