CaseFacts：法律事实核查与先例检索的基准
CaseFacts: A Benchmark for Legal Fact-Checking and Precedent Retrieval
AI总结：
本文提出CaseFacts基准，用于验证普通法律主张与美国最高法院先例之间的关系，通过多阶段流程利用大语言模型生成数据，并发现任务仍具挑战性，开放检索反而降低性能。
摘要：
自动化事实核查大多集中在验证通用知识与静态语料库之间的关系，忽视了法律等高风险领域，其中事实在不断演变且技术复杂。我们引入CaseFacts基准，用于验证普通法律主张与美国最高法院先例之间的关系。不同于现有资源将正式文本映射到正式文本，CaseFacts挑战系统在考虑时间有效性的同时，弥合普通人的断言与技术法律原则之间的语义差距。数据集包含6,294个被归类为支持、反驳或推翻的主张。我们通过多阶段流程构建此基准，利用大语言模型（LLM）生成主张，采用新颖的语义相似性启发式方法来高效识别和验证复杂的法律推翻。使用最先进的LLM进行实验发现，该任务仍然具有挑战性；值得注意的是，通过添加不受限制的网络搜索增强模型性能，相较于闭卷基线表现更差，因为检索到的噪声、非权威性先例。我们发布CaseFacts以促进法律事实验证系统的研究。
英文摘要：
Automated Fact-Checking has largely focused on verifying general knowledge against static corpora, overlooking high-stakes domains like law where truth is evolving and technically complex. We introduce CaseFacts, a benchmark for verifying colloquial legal claims against U.S. Supreme Court precedents. Unlike existing resources that map formal texts to formal texts, CaseFacts challenges systems to bridge the semantic gap between layperson assertions and technical jurisprudence while accounting for temporal validity. The dataset consists of 6,294 claims categorized as Supported, Refuted, or Overruled. We construct this benchmark using a multi-stage pipeline that leverages Large Language Models (LLMs) to synthesize claims from expert case summaries, employing a novel semantic similarity heuristic to efficiently identify and verify complex legal overrulings. Experiments with state-of-the-art LLMs reveal that the task remains challenging; notably, augmenting models with unrestricted web search degrades performance compared to closed-book baselines due to the retrieval of noisy, non-authoritative precedents. We release CaseFacts to spur research into legal fact verification systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
