HALT：通过log-prob作为时间序列进行幻觉评估
HALT: Hallucination Assessment via Log-probs as Time series
AI总结：
HALT通过log-prob时间序列检测幻觉，比Lettuce更高效且兼容专有模型。
摘要：
幻觉仍然是大型语言模型（LLMs）的主要障碍，尤其是在安全关键领域。我们提出了HALT（通过log-prob作为时间序列进行幻觉评估），这是一种轻量级的幻觉检测器，仅利用LLM生成的前20个token的log-probabilities作为时间序列。HALT结合门控循环单元模型和基于熵的特征来学习模型校准偏差，提供了一种极其高效的替代方案，无需大型编码器。与白盒方法不同，HALT不需要访问隐藏状态或注意力图，仅依赖输出log-probabilities。与黑盒方法不同，它在log-probs上操作而不是表面形式文本，这使得它具有更强的领域泛化能力和兼容专有LLMs的能力，而无需访问内部权重。为了评估性能，我们引入了HUB（幻觉检测统一基准），它将先前的数据集整合到十个能力中，涵盖推理任务（算法、常识、数学、符号、代码生成）和通用技能（聊天、数据到文本、问答、摘要、世界知识）。尽管HALT比Lettuce小30倍，但其在HUB上的表现优于经过微调的现代BERT-base编码器，实现了60倍的速度提升。HALT和HUB共同建立了一个有效的框架，用于在各种LLM能力上进行幻觉检测。
英文摘要：
Hallucinations remain a major obstacle for large language models (LLMs), especially in safety-critical domains. We present HALT (Hallucination Assessment via Log-probs as Time series), a lightweight hallucination detector that leverages only the top-20 token log-probabilities from LLM generations as a time series. HALT uses a gated recurrent unit model combined with entropy-based features to learn model calibration bias, providing an extremely efficient alternative to large encoders. Unlike white-box approaches, HALT does not require access to hidden states or attention maps, relying only on output log-probabilities. Unlike black-box approaches, it operates on log-probs rather than surface-form text, which enables stronger domain generalization and compatibility with proprietary LLMs without requiring access to internal weights. To benchmark performance, we introduce HUB (Hallucination detection Unified Benchmark), which consolidates prior datasets into ten capabilities covering both reasoning tasks (Algorithmic, Commonsense, Mathematical, Symbolic, Code Generation) and general purpose skills (Chat, Data-to-Text, Question Answering, Summarization, World Knowledge). While being 30x smaller, HALT outperforms Lettuce, a fine-tuned modernBERT-base encoder, achieving a 60x speedup gain on HUB. HALT and HUB together establish an effective framework for hallucination detection across diverse LLM capabilities.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
