JuDGE：面向中国法律系统的判决书生成基准测试
JuDGE: Benchmarking Judgment Document Generation for Chinese Legal System
AI总结：
本文提出JuDGE基准，用于评估中国法律系统中的判决书生成任务，通过构建真实案件数据集与外部法律语料库，并联合法律专家建立自动化评估框架，验证了少样本学习、微调及多源RAG方法，发现RAG有效但仍有改进空间。
摘要：
本文介绍了JuDGE（Judgment Document Generation Evaluation，判决书生成评估），一个用于评估中国法律系统中判决书生成性能的新颖基准。我们将该任务定义为根据给定的案件事实描述生成完整的法律判决书。为支持该基准，我们构建了一个综合数据集，包含来自真实法律案件的事实描述，并配以相应的完整判决书，作为评估生成文档质量的基准真值。该数据集进一步通过两个外部法律语料库得到增强，为任务提供额外的法律知识：一个由法律法规组成，另一个包含大量过往判决书。我们与法律专业人士合作，建立了一个全面的自动化评估框架，从多个维度评估生成判决书的质量。我们使用通用和法律领域的大语言模型评估了各种基线方法，包括少样本上下文学习、微调以及多源检索增强生成（RAG）方法。实验结果表明，尽管RAG方法能有效提升该任务的性能，但仍有很大的进一步改进空间。所有代码和数据集可在以下地址获取：https://github.com/oneal2000/JuDGE。
英文摘要：
This paper introduces JuDGE (Judgment Document Generation Evaluation), a novel benchmark for evaluating the performance of judgment document generation in the Chinese legal system. We define the task as generating a complete legal judgment document from the given factual description of the case. To facilitate this benchmark, we construct a comprehensive dataset consisting of factual descriptions from real legal cases, paired with their corresponding full judgment documents, which serve as the ground truth for evaluating the quality of generated documents. This dataset is further augmented by two external legal corpora that provide additional legal knowledge for the task: one comprising statutes and regulations, and the other consisting of a large collection of past judgment documents. In collaboration with legal professionals, we establish a comprehensive automated evaluation framework to assess the quality of generated judgment documents across various dimensions. We evaluate various baseline approaches, including few-shot in-context learning, fine-tuning, and a multi-source retrieval-augmented generation (RAG) approach, using both general and legal-domain LLMs. The experimental results demonstrate that, while RAG approaches can effectively improve performance in this task, there is still substantial room for further improvement. All the codes and datasets are available at: https://github.com/oneal2000/JuDGE.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
