FactReasoner：一种用于大语言模型长文本事实性评估的概率方法
FactReasoner: A Probabilistic Approach to Long-Form Factuality Assessment for Large Language Models
AI总结：
针对大语言模型生成长文本时的事实准确性问题，提出基于神经符号的FactReasoner框架，通过分解原子单元、检索外部证据、概率建模逻辑关系评估真实性，在事实精确率和召回率上优于现有方法且开源。
摘要：
大语言模型（LLMs）在生成任务中取得了显著成功，但在确保输出事实准确性方面往往存在不足，从而限制了其在正确性至关重要的实际应用中的可靠性。本文提出FactReasoner，一种新颖的基于神经符号的事实性评估框架，该框架采用概率推理来评估长文本生成响应的真实性。FactReasoner将响应分解为原子单元，从外部知识源检索相关上下文信息，并使用概率编码对这些单元与其上下文之间的逻辑关系（如蕴含、矛盾）进行建模。然后，它估计每个原子单元得到检索证据支持的后验概率。我们在带标签和无标签基准数据集上的实验表明，FactReasoner在事实精确率和召回率方面通常优于最先进的基于提示的方法。我们的开源实现可在以下网址获取：https://github.com/IBM/FactReasoner。
英文摘要：
Large language models (LLMs) have achieved remarkable success in generative tasks, yet they often fall short in ensuring the factual accuracy of their outputs, thus limiting their reliability in real-world applications where correctness is critical. In this paper, we present FactReasoner, a novel neuro-symbolic based factuality assessment framework that employs probabilistic reasoning to evaluate the truthfulness of long-form generated responses. FactReasoner decomposes a response into atomic units, retrieves relevant contextual information from external knowledge sources, and models the logical relationships (e.g., entailment, contradiction) between these units and their contexts using probabilistic encodings. It then estimates the posterior probability that each atomic unit is supported by the retrieved evidence. Our experiments on both labeled and unlabeled benchmark datasets demonstrate that FactReasoner often outperforms state-of-the-art prompt-based methods in terms of factual precision and recall. Our open-source implementation is publicly available at: https://github.com/IBM/FactReasoner.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
