LexKairos：评估大型语言模型的法律时间能力基准
LexKairos: Benchmarking Legal Temporal Capabilities in LLMs
浏览论文内容
中文总结 AI 辅助
本研究提出LexKairos基准，评估LLMs的中文法律时间能力，经多设置测试发现Gemini-3-Flash表现最优，但现有模型在时间敏感任务上仍存局限，相关能力待提升。
中文摘要 AI 辅助
大型语言模型（LLMs）在各类法律任务中展现出强大性能。在法律实践中，时间是决定法规效力、案件进展及程序期限执行的关键概念。然而，现有法律AI基准对法律时间能力的探索仍不充分。为填补这一空白，我们提出LexKairos——一个针对中文法律场景下LLMs时间能力的综合基准，涵盖法规时间知识、案件时间建模、法规-案件时间推理三个维度。LexKairos包含9个来自真实中国司法案例与法规的子任务。我们在多种推理设置（包括标准设置、思维链（CoT）、思考模式）下对8个LLMs进行系统评估。结果显示，Gemini-3-Flash取得整体最强性能，但即便是表现最优的模型，在需要精准时间敏感法规元数据回忆或时限内复杂推理的任务上仍存在明显局限，表明法律时间知识与推理仍是当前LLMs面临的开放挑战。数据与代码可在该https URL获取。
英文摘要
Large language models (LLMs) have demonstrated strong performance across a wide range of legal tasks. In legal practice, time is a critical concept that governs the validity of statutes, the progression of legal cases, and the enforcement of procedural deadlines. However, legal temporal capabilities remain underexplored in existing legal AI benchmarks. To address this gap, we propose LexKairos, a comprehensive benchmark for evaluating the temporal capabilities of LLMs in the Chinese legal context across three dimensions: statutory temporal knowledge, case temporal modeling, and statute-case temporal reasoning. LexKairos comprises nine sub-tasks drawn from real-world Chinese judicial cases and statutes. We conduct systematic evaluations of eight LLMs under multiple inference settings, including vanilla, Chain-of-Thought (CoT), and thinking modes. Our results show that Gemini-3-Flash achieves the strongest overall performance, yet even the best-performing model exhibits notable limitations on tasks demanding precise time-sensitive statutory metadata recall or complex reasoning in time limits, indicating that legal temporal knowledge and reasoning remain open challenges for current LLMs. Data and code are available at https://github.com/thunlp/LexKairos.
发表机构
- University of Oxford（牛津大学）
- Sun Yat-Sen University（中山大学）
- Tsinghua University（清华大学）
机构由 AI 辅助整理，请以论文原文为准。
