FactLens：细粒度事实验证基准测试
FactLens: Benchmarking Fine-Grained Fact Verification
AI总结：
本文提出细粒度事实验证基准FactLens，通过将复杂声明拆分为子声明进行单独验证，以解决LLM幻觉问题并提升验证精度，实验表明其自动化评估器与人类判断高度一致。
摘要：
大型语言模型（LLMs）在语言生成与理解方面展现了令人印象深刻的能力，但其产生幻觉和输出事实错误信息的倾向仍是一个关键局限。为了验证LLM生成的内容及其他来源的声明，传统验证方法通常依赖整体模型为复杂声明分配单一事实性标签，这可能掩盖细微的错误。本文中，我们提倡向细粒度验证转变，即将复杂声明拆分为更小的子声明进行单独验证，从而实现更精确的不准确之处识别、提高透明度并减少证据检索中的歧义。然而，生成子声明带来了挑战，如维持上下文及确保与原始声明的语义等价性。我们引入了FactLens，这是一个用于评估细粒度事实验证的基准，包含子声明质量的指标和自动化评估器。该基准数据经过人工筛选以确保高质量的基准真值。我们的结果显示，自动化FactLens评估器与人类判断之间存在一致性，并探讨了子声明特征对整体验证性能的影响。
英文摘要：
Large Language Models (LLMs) have shown impressive capability in language generation and understanding, but their tendency to hallucinate and produce factually incorrect information remains a key limitation. To verify LLM-generated contents and claims from other sources, traditional verification approaches often rely on holistic models that assign a single factuality label to complex claims, potentially obscuring nuanced errors. In this paper, we advocate for a shift towards fine-grained verification, where complex claims are broken down into smaller sub-claims for individual verification, allowing for more precise identification of inaccuracies, improved transparency, and reduced ambiguity in evidence retrieval. However, generating sub-claims poses challenges, such as maintaining context and ensuring semantic equivalence with respect to the original claim. We introduce FactLens, a benchmark for evaluating fine-grained fact verification, with metrics and automated evaluators of sub-claim quality. The benchmark data is manually curated to ensure high-quality ground truth. Our results show alignment between automated FactLens evaluators and human judgments, and we discuss the impact of sub-claim characteristics on the overall verification performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
