Reviewing contracts is a time-consuming procedure that incurs large expenses to companies and social inequality to those who cannot afford it. In this work, we propose "document-level natural language inference (NLI) for contracts", a novel, real-world application of NLI that addresses such problems. In this task, a system is given a set of hypotheses (such as "Some obligations of Agreement may survive termination.") and a contract, and it is asked to classify whether each hypothesis is "entailed by", "contradicting to" or "not mentioned by" (neutral to) the contract as well as identifying "evidence" for the decision as spans in the contract. We annotated and release the largest corpus to date consisting of 607 annotated contracts. We then show that existing models fail badly on our task and introduce a strong baseline, which (1) models evidence identification as multi-label classification over spans instead of trying to predict start and end tokens, and (2) employs more sophisticated context segmentation for dealing with long documents. We also show that linguistic characteristics of contracts, such as negations by exceptions, are contributing to the difficulty of this task and that there is much room for improvement.
SQuALITY: Building a Long-Document Summarization Dataset the Hard Way
New York University(纽约大学)
摘要数据集通常是通过抓取自然发生的公共领域摘要(几乎总是在难以使用的技术领域)或通过使用近似算法从日常文本中提取它们来组装的,这通常会产生不忠实的摘要。在这项工作中,我们转向一种更慢但更直接的方法来开发摘要基准数据:我们聘请高素质的承包商从头开始阅读故事并编写原始摘要。为了节省阅读时间,我们为每份文件收集了五个摘要,第一个摘要是概述,随后的四个摘要是针对具体问题的。我们使用该协议来收集SQuality,这是一个以问题为中心的摘要数据集,建立在与多项选择数据集QUALITY相同的公共领域短篇小说上(Pang et al.,2021年)。最先进的摘要系统的实验表明,我们的数据集是具有挑战性的,现有的自动评估指标的质量弱指标。
Same Scores, Different Decisions: Evaluating JEV and Language Models for Legal Document Understanding
The University of Tokyo(东京大学);MBZUAI(穆罕默德·本·扎耶德人工智能大学);McGill University(麦吉尔大学);Hong Kong Baptist University(香港浸会大学);Fudan University(复旦大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);UCloud(优刻得);Columbia University(哥伦比亚大学);The University of Hong Kong(香港大学);University of Illinois Chicago(伊利诺伊大学芝加哥分校);Quantell Capital
GRAPH-GRPO-LEX: Contract Graph Modeling and Reinforcement Learning with Group Relative Policy Optimization
Harvard University(哈佛大学);Illinois Tech - Chicago Kent College of Law(伊利诺伊理工学院-芝加哥肯特法学院);CLTDS, Bucerius Law School(CLTDS,布塞里乌斯法学院);Yong Pung How School of Law, Singapore Management University(永丰何法学院,新加坡管理大学);CodeX - The Stanford Center for Legal Informatics, Stanford University(CodeX-斯坦福法律信息中心,斯坦福大学);Georgetown University(乔治城大学);Massachusetts Institute of Technology(麻省理工学院)
The KL3M Data Project: Copyright-Clean Training Resources for Large Language Models
Institute for the Advancement of Legal and Ethical AI (ALEA Institute)(法律与伦理AI推进研究所(ALEA研究所));CodeX—The Stanford Center for Legal Informatics(CodeX——斯坦福法律信息学中心);Illinois Tech—Chicago Kent College of Law(伊利诺伊理工大学芝加哥肯特法学院);Center for Legal Technology & Data Science, Bucerius Law School(布塞留斯法学院法律技术与数据科学中心)