基于信念树传播的LLM幻觉检测概率框架
A Probabilistic Framework for LLM Hallucination Detection via Belief Tree Propagation
AI总结：
针对LLM幻觉检测中现有方法增强陈述单调无结构、信念信息丢失问题，提出BTProp概率框架，通过构建信念树并结合隐马尔可夫树模型整合连续信念分数，在多个基准上较基线提升3%-9%。
摘要：
本文聚焦于幻觉检测任务，旨在判断LLM生成陈述的真实性。为解决该问题，一类流行方法利用LLM在其生成的一组逻辑相关增强陈述中的信念自一致性，此类方法无需外部知识库，可适用于白盒和黑盒LLM。然而，现有多数方法中，增强陈述往往单调且无结构化，难以整合LLM在这些陈述中信念的有意义信息；同时，许多方法使用LLM信念的二值化版本而非连续版本，导致信息大量丢失。为此，本文提出信念树传播（BTProp）这一LLM幻觉检测概率框架。BTProp通过三种分解策略将父陈述递归分解为子陈述，构建逻辑相关陈述的信念树，并建立隐马尔可夫树模型，以原则性方式整合LLM在这些陈述中的信念分数。实验结果显示，该方法在多个幻觉检测基准上较基线提升3%-9%（以AUROC和AUC-PR评估）。代码可在https://github.com/UCSB-NLP-Chang/BTProp获取。
英文摘要：
This paper focuses on the task of hallucination detection, which aims to determine the truthfulness of LLM-generated statements. To address this problem, a popular class of methods utilize the LLM's self-consistencies in its beliefs in a set of logically related augmented statements generated by the LLM, which does not require external knowledge databases and can work with both white-box and black-box LLMs. However, in many existing approaches, the augmented statements tend to be very monotone and unstructured, which makes it difficult to integrate meaningful information from the LLM beliefs in these statements. Also, many methods work with the binarized version of the LLM's belief, instead of the continuous version, which significantly loses information. To overcome these limitations, in this paper, we propose Belief Tree Propagation (BTProp), a probabilistic framework for LLM hallucination detection. BTProp introduces a belief tree of logically related statements by recursively decomposing a parent statement into child statements with three decomposition strategies, and builds a hidden Markov tree model to integrate the LLM's belief scores in these statements in a principled way. Experiment results show that our method improves baselines by 3%-9% (evaluated by AUROC and AUC-PR) on multiple hallucination detection benchmarks. Code is available at https://github.com/UCSB-NLP-Chang/BTProp.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
