Understanding the text in legal documents can be challenging due to their complex structure and the inclusion of domain-specific jargon. Laws and regulations are often crafted in such a manner that engagement with them requires formal training, potentially leading to vastly different interpretations of the same texts. Linguistic complexity is an important contributor to the difficulties experienced by readers. Simplifying texts could enhance comprehension across a broader audience, not just among trained professionals. Various metrics have been developed to measure document readability. Therefore, we adopted a systematic review approach to examine the linguistic and readability metrics currently employed for legal and regulatory texts. A total of 3566 initial papers were screened, with 34 relevant studies found and further assessed. Our primary objective was to identify which current metrics were applied for evaluating readability within the legal field. Sixteen different metrics were identified, with the Flesch-Kincaid Grade Level being the most frequently used method. The majority of studies (73.5%) were found in the domain of "informed consent forms". From the analysis, it is clear that not all legal domains are well represented in terms of readability metrics and that there is a further need to develop more consensus on which metrics should be applied for legal documents.
Jožef Stefan Institute（约热夫·斯泰凡研究所）；Jožef Stefan International Postgraduate School（约热夫·斯泰凡国际研究生院）；University of Ljubljana（卢布尔雅那大学）
We present a set of novel neural supervised and unsupervised approaches for determining the readability of documents. In the unsupervised setting, we leverage neural language models, whereas in the supervised setting, three different neural classification architectures are tested. We show that the proposed neural unsupervised approach is robust, transferable across languages and allows adaptation to a specific readability task and data set. By systematic comparison of several neural architectures on a number of benchmark and new labelled readability datasets in two languages, this study also o…
Athens University of Economics and Business（雅典经济与商业大学）；NCSR “Demokritos”（德谟克利特国家科学研究中心）；University of Sheffield（谢菲尔德大学）
BERT has achieved impressive performance in several NLP tasks. However, there has been limited investigation on its adaptation guidelines in specialised domains. Here we focus on the legal domain, where we explore several approaches for applying BERT models to downstream legal tasks, evaluating on multiple datasets. Our findings indicate that the previous guidelines for pre-training and fine-tuning, often blindly followed, do not always generalize well in the legal domain. Thus we propose a systematic investigation of the available strategies when applying BERT in specialised domains. These ar…
IIT Kharagpur（印度理工学院卡哈拉格普尔分校）；L3S Research Center（L3S研究中心）
Automatic summarization of legal case documents is an important and practical challenge. Apart from many domain-independent text summarization algorithms that can be used for this purpose, several algorithms have been developed specifically for summarizing legal case documents. However, most of the existing algorithms do not systematically incorporate domain knowledge that specifies what information should ideally be present in a legal case document summary. To address this gap, we propose an unsupervised summarization algorithm DELSumm which is designed to systematically incorporate guideline…
GreekBarBench: A Challenging Benchmark for Free-Text Legal Reasoning and Citations
Department of Informatics, Athens University of Economics and Business（信息学院，雅典经济与商业大学）；Archimedes, Athena Research Center（阿提卡研究中心-阿基米德）；Athena Research Center（阿提卡研究中心）；University of Sheffield（谢菲尔德大学）
Efficient Prompt Optimisation for Legal Text Classification with Proxy Prompt Evaluator
Technical University of Munich（慕尼黑技术大学）；Department of Computer Science, University of Copenhagen（哥本哈根大学计算机科学系）；Faculty of Law, University of Copenhagen（哥本哈根大学法学院）
Large Language Models Meet Legal Artificial Intelligence: A Survey
School of Computer Science and Engineering, Sun Yat-sen University（中山大学计算机科学与工程学院）；China Mobile Internet Co., Ltd.（中国移动互联网有限公司）；School of Computer Science, South China Normal University（华南师范大学计算机科学学院）
A Comprehensive Evaluation of Large Language Models on Legal Judgment Prediction
National University of Singapore（新加坡国立大学）；Singapore Management University（新加坡管理大学）；Institute of Dataspace, Hefei Comprehensive National Science Center（合肥综合性国家科学中心数据空间研究院）；University of Science and Technology of China（中国科学技术大学）
University of York（约克大学）；University of Sheffield（谢菲尔德大学）
Natural language processing (NLP) methods for analyzing legal text offer legal scholars and practitioners a range of tools allowing to empirically analyze law on a large scale. However, researchers seem to struggle when it comes to identifying ethical limits to using NLP systems for acquiring genuine insights both about the law and the systems' predictive capacity. In this paper we set out a number of ways in which to think systematically about such issues. We place emphasis on three crucial normative parameters which have, to the best of our knowledge, been underestimated by current debates: …
Indian Institute of Technology Kharagpur（印度理工学院克勒格布尔分校）；Tata Research Development and Design Centre (TRDDC)（塔塔研究开发与设计中心（TRDDC））；Data61, CSIRO（澳大利亚联邦科学与工业研究组织 Data61）；Cyber Security CRC（网络安全合作研究中心）
Computing similarity between two legal documents is an important and challenging task in the domain of Legal Information Retrieval. Finding similar legal documents has many applications in downstream tasks, including prior-case retrieval, recommendation of legal articles, and so on. Prior works have proposed two broad ways of measuring similarity between legal documents - analyzing the precedent citation network, and measuring similarity based on textual content similarity measures. But there has not been a comprehensive comparison of these existing methods on a common platform. In this paper,…
German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心（DFKI））
Clinical text processing has gained more and more attention in recent years. The access to sensitive patient data, on the other hand, is still a big challenge, as text cannot be shared without legal hurdles and without removing personal information. There are many techniques to modify or remove patient related information, each with different strengths. This paper investigates the influence of different anonymization techniques on the performance of ML models using multiple datasets corresponding to five different NLP tasks. Several learnings and recommendations are presented. This work confir…
SinhaLegal: A Benchmark Corpus for Information Extraction and Analysis in Sinhala Legislative Texts
School of Computing, Informatics Institute of Technology（计算学院，信息学院技术研究所）；Department of Computer Science & Engineering, University of Moratuwa（计算机科学与工程系，莫图瓦大学）
Technical University of Darmstadt（达姆施塔特工业大学）；Hessian Center for AI (hessian.AI)（黑森人工智能中心）
The shift towards publicly available text sources has enabled language processing at unprecedented scale, yet leaves under-serviced the domains where public and openly licensed data is scarce. Proactively collecting text data for research is a viable strategy to address this scarcity, but lacks systematic methodology taking into account the many ethical, legal and confidentiality-related aspects of data collection. Our work presents a case study on proactive data collection in peer review -- a challenging and under-resourced NLP domain. We outline ethical and legal desiderata for proactive dat…
Evaluating LLM-based Approaches to Legal Citation Prediction: Domain-specific Pre-training, Fine-tuning, or RAG? A Benchmark and an Australian Law Case Study
Department of Data Science & AI, Monash University（数据科学与人工智能系，莫纳什大学）；Faculty of Law, Monash University（法学院，莫纳什大学）
When Large Language Models Meet Law: Dual-Lens Taxonomy, Technical Advances, and Ethical Governance
School of Law, China University of Political Science and Law（中国政法大学法学院）；Zhejiang University of Finance and Economics Dongfang College（浙江财经大学东洋学院）；School of Information Management for Law, China University of Political Science and Law（中国政法大学信息管理法学院）；Department of Artificial Intelligence, Chung-Ang University（成均馆大学人工智能系）；Department of Data Science and Artificial Intelligence, The Hong Kong Polytechnic University（香港理工大学数据科学与人工智能系）
