预测编码与信息瓶颈用于大语言模型中的幻觉检测
Predictive Coding and Information Bottleneck for Hallucination Detection in Large Language Models
AI总结：
本研究提出了一种结合预测编码和信息瓶颈的轻量级框架，用于高效检测大语言模型中的幻觉，实现了比传统方法更高的准确率和效率。
摘要：
大语言模型（LLMs）中的幻觉——生成内容看似合理但事实不准确——仍然是高风险应用中的关键障碍。当前的检测方法通常依赖于计算成本高昂的外部检索循环或需要700亿参数的不可解释的黑箱LLM判断者。在本工作中，我们介绍了[模型名称]，一种结合神经科学启发的信号设计与监督机器学习的混合检测框架。我们提取了基于预测编码（量化惊喜与内部先验的对比）和信息瓶颈（测量扰动下的信号保留）的可解释信号。通过系统性消融分析，我们展示了三个关键改进：实体聚焦吸收（集中于高价值标记）、上下文一致性（测量接地强度）以及可证伪性分数（检测自信但矛盾的声明）。在HaluBench（n=200，完全平衡）上评估，我们的理论指导基线达到0.8017 AUROC。BASE监督模型达到0.8274 AUROC，而改进的特征将性能提升到0.8669 AUROC（4.95%提升），在不同架构上均显示出一致的改进。这种竞争性性能是在训练数据仅为Lynx（200 vs 15000样本）的75分之一、推理速度加快1000倍（5毫秒 vs 5秒）的情况下实现的，并且仍然完全可解释。关键的是，我们报告了一个负面结果：理性化信号无法区分幻觉，表明LLM为假前提生成连贯的推理（
英文摘要：
Hallucinations in Large Language Models (LLMs) -- generations that are plausible but factually unfaithful -- remain a critical barrier to high-stakes deployment. Current detection methods typically rely on computationally expensive external retrieval loops or opaque black-box LLM judges requiring 70B+ parameters. In this work, we introduce [Model Name], a hybrid detection framework that combines neuroscience-inspired signal design with supervised machine learning. We extract interpretable signals grounded in Predictive Coding (quantifying surprise against internal priors) and the Information Bottleneck (measuring signal retention under perturbation). Through systematic ablation, we demonstrate three key enhancements: Entity-Focused Uptake (concentrating on high-value tokens), Context Adherence (measuring grounding strength), and Falsifiability Score (detecting confident but contradictory claims). Evaluating on HaluBench (n=200, perfectly balanced), our theory-guided baseline achieves 0.8017 AUROC. BASE supervised models reach 0.8274 AUROC, while IMPROVED features boost performance to 0.8669 AUROC (4.95% gain), demonstrating consistent improvements across architectures. This competitive performance is achieved while using 75x less training data than Lynx (200 vs 15,000 samples), 1000x faster inference (5ms vs 5s), and remaining fully interpretable. Crucially, we report a negative result: the Rationalization signal fails to distinguish hallucinations, suggesting that LLMs generate coherent reasoning for false premises ("Sycophancy"). This work demonstrates that domain knowledge encoded in signal architecture provides superior data efficiency compared to scaling LLM judges, achieving strong performance with lightweight (less than 1M parameter), explainable models suitable for production deployment.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
