AGB-DE：用于德国消费者合同条款自动化法律评估的语料库
AGB-DE: A Corpus for the Automated Legal Assessment of Clauses in German Consumer Contracts
AI总结：
本文构建了由法律专家标注的 AGB-DE 德国消费者合同条款语料库，并以 SVM、微调开放语言模型和 GPT-3.5 建立无效条款检测基线，揭示复杂条款解释是核心难点。
摘要：
法律任务和数据集常被用作衡量语言模型能力的基准。然而，公开可用的标注数据集十分稀少。本文介绍了 AGB-DE，这是一个包含来自德国消费者合同的 3,764 条条款的语料库，这些条款已由法律专家完成标注和法律评估。除数据外，我们还为检测潜在无效条款的任务提出了首个基线，比较了 SVM 基线、三个经过微调的开放语言模型以及 GPT-3.5 的性能。结果表明该任务具有挑战性：没有任何方法的 F1 分数超过 0.54。尽管微调模型在精确率方面通常表现更好，但 GPT-3.5 在召回率方面优于其他方法。错误分析表明，主要挑战之一可能在于正确解释复杂条款，而不是界定何为允许、何为不允许的决策边界。
英文摘要：
Legal tasks and datasets are often used as benchmarks for the capabilities of language models. However, openly available annotated datasets are rare. In this paper, we introduce AGB-DE, a corpus of 3,764 clauses from German consumer contracts that have been annotated and legally assessed by legal experts. Together with the data, we present a first baseline for the task of detecting potentially void clauses, comparing the performance of an SVM baseline with three fine-tuned open language models and the performance of GPT-3.5. Our results show the challenging nature of the task, with no approach exceeding an F1-score of 0.54. While the fine-tuned models often performed better with regard to precision, GPT-3.5 outperformed the other approaches with regard to recall. An analysis of the errors indicates that one of the main challenges could be the correct interpretation of complex clauses, rather than the decision boundaries of what is permissible and what is not.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
