ThinknCheck：基于紧凑、推理驱动和可解释模型的 grounded claim 验证
ThinknCheck: Grounded Claim Verification with Compact, Reasoning-Driven, and Interpretable Models
- University of Pennsylvania（宾夕法尼亚大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
ThinknCheck 通过生成结构化推理和二元判断实现 grounded claim 验证，其在 LLMAggreFact 上达到 78.1% 的平衡准确率，优于 MiniCheck-7B，且在 SciFact 上提升 14.7 个百分点。
AI中文摘要：
我们提出了 ThinknCheck，一种 1B 参数的验证器，首先生成简短的结构化推理然后给出二元判断。我们构建了 LLMAggreFact-Think，一个 24.1k 个推理增强的训练集，基于 LLMAggreFact，并微调 4 位 Gemma3 模型以遵循此格式。在 LLMAggreFact 上，ThinknCheck 达到 78.1% 的平衡准确率（BAcc），超过 MiniCheck-7B（77.4%）且参数少 7 倍；去除推理步骤会降低 BAcc 到 57.5%。在 SciFact 上，ThinknCheck 达到 64.7% 的 BAcc，比 MiniCheck-7B 提高 14.7 个百分点。相比之下，基于基础 Gemma3-1B 的零样本链式思维损害了准确性，而使用简单格式+准确性奖励的偏好优化效果不如监督推理。为了探测后者，我们引入了 GSMClaims 和一个领域专用的变体 ThinknCheck-Science，其在多个基准测试中表现优异，包括在 GSMClaims 上达到 61.0% 的准确率。总体而言，显式、监督的推理使紧凑的验证器在保持资源效率和可解释性的同时具有竞争力。
英文摘要：
We present ThinknCheck, a 1B-parameter verifier for grounded claim verification that first produces a short, structured rationale and then a binary verdict. We construct LLMAggreFact-Think, a 24.1k reasoning-augmented training set derived from LLMAggreFact, and fine-tune a 4-bit Gemma3 model to follow this format. On LLMAggreFact, ThinknCheck attains 78.1 balanced accuracy (BAcc), surpassing MiniCheck-7B (77.4) with 7x fewer parameters; removing the reasoning step reduces BAcc to 57.5. On SciFact, ThinknCheck reaches 64.7 BAcc, a +14.7 absolute gain over MiniCheck-7B. By contrast, zero-shot chain-of-thought on the base Gemma3-1B harms accuracy relative to direct answers, and preference optimization with a simple format+accuracy reward underperforms supervised reasoning. To probe the latter, we introduce GSMClaims and a domain-specialized variant, ThinknCheck-Science, which improves across benchmarks, including 61.0\% accuracy on GSMClaims. Overall, explicit, supervised reasoning enables compact verifiers that are competitive while remaining resource-efficient and interpretable.
