Large Transformer-based language models such as BERT have led to broad performance improvements on many NLP tasks. Domain-specific variants of these models have demonstrated excellent performance on a variety of specialised tasks. In legal NLP, BERT-based models have led to new state-of-the-art results on multiple tasks. The exploration of these models has demonstrated the importance of capturing the specificity of the legal language and its vocabulary. However, such approaches suffer from high computational costs, leading to a higher ecological impact and lower accessibility. Our findings, focusing on English language legal text, show that lightweight LSTM-based Language Models are able to capture enough information from a small legal text pretraining corpus and achieve excellent performance on short legal text classification tasks. This is achieved with a significantly reduced computational overhead compared to BERT-based models. However, our method also shows degraded performance on a more complex task, multi-label classification of longer documents, highlighting the limitations of this lightweight approach.
Network public opinion analysis is obtained by a combination of natural language processing (NLP) and public opinion supervision, and is crucial for monitoring public mood and trends. Therefore, network public opinion analysis can identify and solve potential and budding social problems. This study aims to realize an analysis of Chinese sentiment in social media reviews using a long short-term memory network (LSTM) model. The dataset was obtained from Sina Weibo using a web crawler and was cleaned with Pandas. First, Chinese comments regarding the legal sentencing in of Tangshan attack and Jia…
University of Moratuwa（莫拉图瓦大学）；University of Colombo（科伦坡大学）
Analyzing the sentiments of legal opinions available in Legal Opinion Texts can facilitate several use cases such as legal judgement prediction, contradictory statements identification and party-based sentiment analysis. However, the task of developing a legal domain specific sentiment annotator is challenging due to resource constraints such as lack of domain specific labelled data and domain expertise. In this study, we propose novel techniques that can be used to develop a sentiment annotator for the legal domain while minimizing the need for manual annotations of data.
A document which elaborates opinions and arguments related to the previous court cases is known as a legal opinion text. Lawyers and legal officials have to spend considerable effort and time to obtain the required information manually from those documents when dealing with new legal cases. Hence, it provides much convenience to those individuals if there is a way to automate the process of extracting information from legal opinion texts. Party-based sentiment analysis will play a key role in the automation system by identifying opinion values with respect to each legal parties in legal texts.
Democratizing the medieval English legal tradition
University of Chicago（芝加哥大学）；California State University（加州州立大学）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；Carnegie Mellon University（卡内基梅隆大学）；Unaffiliated（无隶属机构）
Towards Intelligent Legal Document Analysis: CNN-Driven Classification of Case Law Texts
Department of Computer Science and Engineering（计算机科学与工程系）；American International University--Bangladesh（国际大学（孟加拉国））；Prime University（Prime大学）；Shajalal University of Science and Technology（沙贾拉尔科学与技术大学）；George Mason University（乔治·梅森大学）；AI and Big Data Department（人工智能与大数据部门）；Woosong University（沃宗大学）
Legal artificial intelligence (LegalAI) aims to benefit legal systems with the technology of artificial intelligence, especially natural language processing (NLP). Recently, inspired by the success of pre-trained language models (PLMs) in the generic domain, many LegalAI researchers devote their effort to apply PLMs to legal tasks. However, utilizing PLMs to address legal tasks is still challenging, as the legal documents usually consist of thousands of tokens, which is far longer than the length that mainstream PLMs can process. In this paper, we release the Longformer-based pre-trained langu…
Seeking legal advice is often expensive. Recent advancements in machine learning for solving complex problems can be leveraged to help make legal services more accessible to the public. However, real-life applications encounter significant challenges. State-of-the-art language models are growing increasingly large, making parameter-efficient learning increasingly important. Unfortunately, parameter-efficient methods perform poorly with small amounts of data, which are common in the legal domain (where data labelling costs are high). To address these challenges, we propose parameter-efficient l…
DeliLaw: A Chinese Legal Counselling System Based on a Large Language Model
University of Chinese Academy of Sciences（中国科学院大学）；Shenzhen Institute of Advanced Technology, CAS（中国科学院深圳先进技术研究院）；Hebei University（河北大学）；University of Science and Technology of China（中国科学技术大学）；South University of Science and Technology of China（南方科技大学）
Athens University of Economics and Business（雅典经济与商业大学）；University of Copenhagen（哥本哈根大学）；Cognitiv+
Pre-trained Transformers currently dominate most NLP tasks. They impose, however, limits on the maximum input length (512 sub-words in BERT), which are too restrictive in the legal domain. Even sparse-attention models, such as Longformer and BigBird, which increase the maximum input length to 4,096 sub-words, severely truncate texts in three of the six datasets of LexGLUE. Simpler linear classifiers with TF-IDF features can handle texts of any length, require far less resources to train and deploy, but are usually outperformed by pre-trained Transformers. We explore two directions to cope with…
DISC-LawLLM: Fine-tuning Large Language Models for Intelligent Legal Services
Fudan University（复旦大学）；Huazhong University of Science and Technology（华中科技大学）；New York University Shanghai（上海纽约大学）；Northwest University of Political and Law（西北政法大学）
Legal Syllogism Prompting: Teaching Large Language Models for Legal Judgment Prediction
Peking University Law School（北京大学法学院）；Institute for Artificial Intelligence, Peking University（北京大学人工智能研究院）；PKU-WUHAN Institute for Artificial Intelligence（北大-武汉人工智能研究院）
Deep learning-based Natural Language Processing methods, especially transformers, have achieved impressive performance in the last few years. Applying those state-of-the-art NLP methods to legal activities to automate or simplify some simple work is of great value. This work investigates the value of domain adaptive pre-training and language adapters in legal NLP tasks. By comparing the performance of language models with domain adaptive pre-training on different tasks and different dataset splits, we show that domain adaptive pre-training is only helpful with low-resource downstream tasks, th…
Enhancing Pre-Trained Language Models with Sentence Position Embeddings for Rhetorical Roles Recognition in Legal Opinions
Nantes Université（南特大学）；École Centrale Nantes（南特中央理工学院）；CNRS（法国国家科学研究中心）；LS2N（南特数字科学实验室）；Peoples' Friendship University of Russia (RUDN University)（俄罗斯人民友谊大学（RUDN大学））；Joint Institute for Nuclear Research（联合核研究所）；Vrije Universiteit Amsterdam（阿姆斯特丹自由大学）；University of Skövde（舍夫德大学）
Athens University of Economics and Business（雅典经济与商业大学）；NCSR “Demokritos”（德谟克利特国家科学研究中心）；University of Sheffield（谢菲尔德大学）
BERT has achieved impressive performance in several NLP tasks. However, there has been limited investigation on its adaptation guidelines in specialised domains. Here we focus on the legal domain, where we explore several approaches for applying BERT models to downstream legal tasks, evaluating on multiple datasets. Our findings indicate that the previous guidelines for pre-training and fine-tuning, often blindly followed, do not always generalize well in the legal domain. Thus we propose a systematic investigation of the available strategies when applying BERT in specialised domains. These ar…
The effectiveness of the BERT model on multiple linguistic tasks has been well documented. On the other hand, its potentials for narrow and specific domains such as Legal, have not been fully explored. In this paper, we examine how BERT can be used in the Arabic legal domain and try customizing this language model for several downstream tasks using several different domain-relevant training and testing datasets to train BERT from scratch. We introduce the AraLegal-BERT, a bidirectional encoder Transformer-based model that have been thoroughly tested and carefully optimized with the goal to amp…
Legal Prompt Engineering (LPE) or Legal Prompting is a process to guide and assist a large language model (LLM) with performing a natural legal language processing (NLLP) skill. Our goal is to use LPE with LLMs over long legal documents for the Legal Judgement Prediction (LJP) task. We investigate the performance of zero-shot LPE for given facts in case-texts from the European Court of Human Rights (in English) and the Federal Supreme Court of Switzerland (in German, French and Italian). Our results show that zero-shot LPE is better compared to the baselines, but it still falls short compared …
Zhejiang University（浙江大学）；AZFT Joint Lab for Knowledge Engine（AZFT知识引擎联合实验室）；Hangzhou Innovation Center, Zhejiang University（浙江大学杭州创新中心）；Alibaba Group（阿里巴巴集团）
Recent years have witnessed the prosperity of legal artificial intelligence with the development of technologies. In this paper, we propose a novel legal application of legal provision prediction (LPP), which aims to predict the related legal provisions of affairs. We formulate this task as a challenging knowledge graph completion problem, which requires not only text understanding but also graph reasoning. To this end, we propose a novel text-guided graph reasoning approach. We collect amounts of real-world legal provision data from the Guangdong government service website and construct a leg…
University of Michigan（密歇根大学）；University of São Paulo（圣保罗大学）；Tikal Tech（蒂卡尔科技公司）；Latam Datalab Serasa Experian（Serasa Experian拉美数据实验室）
We present and make available pre-trained language models (Phraser, Word2Vec, Doc2Vec, FastText, and BERT) for the Brazilian legal language, a Python package with functions to facilitate their use, and a set of demonstrations/tutorials containing some applications involving them. Given that our material is built upon legal texts coming from several Brazilian courts, this initiative is extremely helpful for the Brazilian legal field, which lacks other open and specific tools and language models. Our main objective is to catalyze the use of natural language processing tools for legal texts analy…
Recent developments in Natural Language Processing have led to the introduction of state-of-the-art Neural Language Models, enabled with unsupervised transferable learning, using different pretraining objectives. While these models achieve excellent results on the downstream NLP tasks, various domain adaptation techniques can improve their performance on domain-specific tasks. We compare and analyze the pretrained Neural Language Models, XLNet (autoregressive), and BERT (autoencoder) on the Legal Tasks. Results show that XLNet Model performs better on our Sequence Classification task of Legal …
Recent advances in the area of legal information systems have led to a variety of applications that promise support in processing and accessing legal documents. Unfortunately, these applications have various limitations, e.g., regarding scope or extensibility. Furthermore, we do not observe a trend towards open access in digital libraries in the legal domain as we observe in other domains, e.g., economics of computer science. To improve open access in the legal domain, we present our approach for an open source platform to transparently process and access Legal Open Data. This enables the sust…
LawGPT: A Chinese Legal Knowledge-Enhanced Large Language Model
National Key Laboratory for Novel Software Technology, Nanjing University（南京大学计算机软件新技术国家重点实验室）；School of Artifical Intelligence, Nanjing University（南京大学人工智能学院）；School of Intelligence Science and Technology, Nanjing University（南京大学智能科学与技术学院）
Leveraging Large Language Models for Learning Complex Legal Concepts through Storytelling
University of Virginia School of Law（弗吉尼亚大学法学院）；Allen Institute for AI（艾伦人工智能研究所）；Massachusetts Institute of Technology（麻省理工学院）；Harvard Law School（哈佛法学院）；University of Tokyo（东京大学）
Japan Advanced Institute of Science and Technology（日本北陆先端科学技术大学院大学）
In recent years, thanks to breakthroughs in neural network techniques especially attentive deep learning models, natural language processing has made many impressive achievements. However, automated legal word processing is still a difficult branch of natural language processing. Legal sentences are often long and contain complicated legal terminologies. Hence, models that work well on general documents still face challenges in dealing with legal documents. We have verified the existence of this problem with our experiments in this work. In this dissertation, we selectively present the main ac…
