FaStfact：在大型语言模型中实现更快、更强有力的长文本事实性评估
FaStfact: Faster, Stronger Long-Form Factuality Evaluations in LLMs
AI总结：
本文提出FaStfact评估框架，通过块级声明提取与基于置信度的预验证，并利用文档级证据选择性检索，解决了LLM长文本事实性评估中的效率与可靠性问题，实现了与人类评估高度一致的快速准确评估。
摘要：
由于效率瓶颈和可靠性问题，评估大型语言模型（LLMs）长文本生成的事实性仍然具有挑战性。先前的努力尝试通过将文本分解为声明、搜索证据和验证声明来实现这一点，但存在关键缺陷：（1）由于过于复杂的流水线组件导致效率低下，以及（2）由于不准确的声明集和不充分的证据导致无效。为了解决这些限制，我们提出了FaStfact，这是一个在现有基线中实现与人类评估最高度一致以及时间/Token效率的评估框架。FaStfact首先采用块级声明提取，并结合基于置信度的预验证，在确保可靠性的同时显著降低了时间和Token成本。对于搜索和验证，它从抓取的网页中收集文档级证据，并在验证期间选择性地检索这些证据。基于标注基准FaStfact-Bench的大量实验证明了FaStfact在高效且有效地评估长文本事实性方面的可靠性。代码、基准数据和注释接口工具可在https://github.com/Yingjia-Wan/FaStfact获取。
英文摘要：
Evaluating the factuality of long-form generations from Large Language Models (LLMs) remains challenging due to efficiency bottlenecks and reliability concerns. Prior efforts attempt this by decomposing text into claims, searching for evidence, and verifying claims, but suffer from critical drawbacks: (1) inefficiency due to overcomplicated pipeline components, and (2) ineffectiveness stemming from inaccurate claim sets and insufficient evidence. To address these limitations, we propose \textbf{FaStfact}, an evaluation framework that achieves the highest alignment with human evaluation and time/token efficiency among existing baselines. FaStfact first employs chunk-level claim extraction integrated with confidence-based pre-verification, significantly reducing the time and token cost while ensuring reliability. For searching and verification, it collects document-level evidence from crawled web-pages and selectively retrieves it during verification. Extensive experiments based on an annotated benchmark \textbf{FaStfact-Bench} demonstrate the reliability of FaStfact in both efficiently and effectively evaluating long-form factuality. Code, benchmark data, and annotation interface tool are available at https://github.com/Yingjia-Wan/FaStfact.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
