通过结合离线一致性检查的探针训练实现可迁移且高效的非事实内容检测
Transferable and Efficient Non-Factual Content Detection via Probe Training with Offline Consistency Checking
AI总结：
本文提出 PINOSE，利用离线自一致性检查结果训练事实性探针，无需人工标注即可跨分布检测非事实内容，并降低在线多次采样的计算开销。
摘要：
检测非事实内容一直是提升大型语言模型（LLM）生成内容可信度的长期目标。当前利用人工标注标签训练的事实性探针，在面对分布外内容时迁移能力有限；而在线自一致性检查由于需要生成多个输出，会带来沉重的计算负担。本文提出 PINOSE，它基于离线自一致性检查结果训练探针模型，从而规避对人工标注数据的需求，并实现跨多种数据分布的迁移能力。由于一致性检查过程是离线进行的，PINOSE 减轻了在线一致性验证中生成多个回复所带来的计算负担。此外，该方法在回复解码之前考察内部状态的多个方面，有助于更有效地检测事实性错误。在事实性检测和问答基准上的实验结果表明，PINOSE 取得了优于现有事实性检测方法的结果。我们的代码和数据集已在该匿名仓库上公开发布。
英文摘要：
Detecting non-factual content is a longstanding goal to increase the trustworthiness of large language models (LLMs) generations. Current factuality probes, trained using humanannotated labels, exhibit limited transferability to out-of-distribution content, while online selfconsistency checking imposes extensive computation burden due to the necessity of generating multiple outputs. This paper proposes PINOSE, which trains a probing model on offline self-consistency checking results, thereby circumventing the need for human-annotated data and achieving transferability across diverse data distributions. As the consistency check process is offline, PINOSE reduces the computational burden of generating multiple responses by online consistency verification. Additionally, it examines various aspects of internal states prior to response decoding, contributing to more effective detection of factual inaccuracies. Experiment results on both factuality detection and question answering benchmarks show that PINOSE achieves surpassing results than existing factuality detection methods. Our code and datasets are publicly available on this anonymized repository.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
