发表机构
Thomson Reuters Foundational Research; Imperial College London（汤姆森路透基础研究机构; 帝国理工学院）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本研究构建了首个针对查询端信息不足的法律基准InsufficiencyBench，评估前沿LLM在信息不足的法律查询中识别缺失要素、避免过早结论的能力，发现现有模型表现不佳。
AI 中文摘要
法律人工智能系统正越来越多地用于回答法律问题，但现有的基准假设查询信息是完全明确的。在实践中，用户会省略对法律结果有实质性影响的事实。我们推出InsufficiencyBench，这是首个针对查询端信息不足问题的法律基准，用于评估模型是否能识别查询缺乏法律实质性信息、确定缺失内容，并避免过早得出结论。我们对三种结构性失败模式（切换、门控和致命前提）下的八个典型缺失要素类别进行了分类，构建了202个基准条目（58个基础查询、144个缺失变体），涵盖六个法律领域和24个美国司法管辖区，且由执业律师标注。对十个前沿模型的评估显示，没有模型在缺失要素识别上超过F2值0.46，中位召回率为0.44。模型要么不加区分地规避，要么在虚构假设下沉默回答，没有模型能同时对缺失查询识别并限定回应，同时直接处理完整查询。
英文摘要
Legal AI systems are increasingly used to answer legal questions, yet existing benchmarks assume queries arrive fully specified. In practice, users omit facts that materially determine the legal outcome. We introduce InsufficiencyBench, the first legal benchmark targeting query-side insufficiency: whether a model recognizes when a query lacks legally material information, identifies what is missing, and refrains from premature conclusions. We formalize a taxonomy of eight canonical missing-element categories across three structural failure modes---switch, gating, and fatal prerequisite--- and construct 202 benchmark items (58 base queries, 144 deficient variants) spanning six legal domains and 24 US jurisdictions and annotated by practising attorneys. Evaluating ten frontier models, we find that no model exceeds F2 = 0.46 on missing-element identification and that the median recall is 0.44. Models either hedge indiscriminately or answer silently under fabricated presumptions. No model both identifies and qualifies responses to deficient queries while directly addressing complete ones.
Comments10 pages, Best Paper Honorable Mention at ICML AI4Law 2026
