DecMetrics：面向大语言模型事实一致性输出的结构化主张分解评分方法
DecMetrics: Structured Claim Decomposition Scoring for Factually Consistent LLM Outputs
AI总结：
针对当前主张分解研究重生成方法、轻分解质量评估的缺口，该研究提出包含三项指标的DecMetrics评估框架，并以此为奖励函数优化轻量级分解模型，旨在设立主张分解基准，提升事实核查系统的可靠性与有效性。
摘要：
主张分解通过将复杂主张拆解为更简单的原子组成部分并识别其中的非事实性元素，在事实核查流程中发挥着关键作用。尽管其重要性显著，当前研究主要聚焦于分解的生成式方法，对这些分解后原子主张的质量评估重视不足。为填补这一空白，我们提出了DecMetrics，它包含三项全新指标：COMPLETENESS（完整性）、CORRECTNESS（正确性）以及SEMANTIC ENTROPY（语义熵），旨在自动评估分解模型生成的主张质量。借助这些指标，我们开发了一个轻量级主张分解模型，通过将这些指标整合为奖励函数来优化其性能。通过自动评估，我们的方法旨在为主张分解设立基准，提升事实核查系统的可靠性与有效性。
英文摘要：
Claim decomposition plays a crucial role in the fact-checking process by breaking down complex claims into simpler atomic components and identifying their unfactual elements. Despite its importance, current research primarily focuses on generative methods for decomposition, with insufficient emphasis on evaluating the quality of these decomposed atomic claims. To bridge this gap, we introduce \textbf{DecMetrics}, which comprises three new metrics: \texttt{COMPLETENESS}, \texttt{CORRECTNESS}, and \texttt{SEMANTIC ENTROPY}, designed to automatically assess the quality of claims produced by decomposition models. Utilizing these metrics, we develop a lightweight claim decomposition model, optimizing its performance through the integration of these metrics as a reward function. Through automatic evaluation, our approach aims to set a benchmark for claim decomposition, enhancing both the reliability and effectiveness of fact-checking systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
