Methods for Computing Legal Document Similarity: A Comparative Study
Indian Institute of Technology Kharagpur（印度理工学院克勒格布尔分校）；Tata Research Development and Design Centre (TRDDC)（塔塔研究开发与设计中心（TRDDC））；Data61, CSIRO（澳大利亚联邦科学与工业研究组织 Data61）；Cyber Security CRC（网络安全合作研究中心）
Computing similarity between two legal documents is an important and challenging task in the domain of Legal Information Retrieval. Finding similar legal documents has many applications in downstream tasks, including prior-case retrieval, recommendation of legal articles, and so on. Prior works have proposed two broad ways of measuring similarity between legal documents - analyzing the precedent citation network, and measuring similarity based on textual content similarity measures. But there has not been a comprehensive comparison of these existing methods on a common platform. In this paper, we perform the first systematic analysis of the existing methods. In addition, we explore two promising new similarity computation methods - one text-based and the other based on network embeddings, which have not been considered till now.
We describe a dataset developed for Named Entity Recognition in German federal court decisions. It consists of approx. 67,000 sentences with over 2 million tokens. The resource contains 54,000 manually annotated entities, mapped to 19 fine-grained semantic classes: person, judge, lawyer, country, city, street, landscape, organization, company, institution, court, brand, law, ordinance, European legal norm, regulation, contract, court decision, and legal literature. The legal documents were, furthermore, automatically annotated with more than 35,000 TimeML-based time expressions. The dataset, w…
University of Moratuwa（莫拉图瓦大学）；University of London（伦敦大学）
Domain specific information retrieval process has been a prominent and ongoing research in the field of natural language processing. Many researchers have incorporated different techniques to overcome the technical and domain specificity and provide a mature model for various domains of interest. The main bottleneck in these studies is the heavy coupling of domain experts, that makes the entire process to be time consuming and cumbersome. In this study, we have developed three novel models which are compared against a golden standard generated via the on line repositories provided, specificall…
Attentive Deep Neural Networks for Legal Document Retrieval
Japan Advanced Institute of Science and Technology（日本北陆先端科学技术大学院大学）；National Institute of Informatics（日本国立信息学研究所）；Posts and Telecommunications Institute of Technology（邮电技术学院）
In the legal domain it is important to differentiate between words in general, and afterwards to link the occurrences of the same entities. The topic to solve these challenges is called Named-Entity Linking (NEL). Current supervised neural networks designed for NEL use publicly available datasets for training and testing. However, this paper focuses especially on the aspect of applying transfer learning approach using networks trained for NEL to legal documents. Experiments show consistent improvement in the legal datasets that were created from the European Union law in the scope of this rese…
CDJUR-BR -- A Golden Collection of Legal Document from Brazilian Justice with Fine-Grained Named Entities
Universidade de Fortaleza（福塔莱萨大学）；Instituto Federal de Educação, Ciência e Tecnologia do Ceará（塞阿拉联邦教育、科学与技术学院）；Tribunal de Justiça do Estado do Ceará（塞阿拉州法院）
Large, pre-trained transformer models like BERT have achieved state-of-the-art results on document understanding tasks, but most implementations can only consider 512 tokens at a time. For many real-world applications, documents can be much longer, and the segmentation strategies typically used on longer documents miss out on document structure and contextual information, hurting their results on downstream tasks. In our work on legal agreements, we find that visual cues such as layout, style, and placement of text in a document are strong features that are crucial to achieving an acceptable l…
The growing complexity of legal cases has lead to an increasing interest in legal information retrieval systems that can effectively satisfy user-specific information needs. However, such downstream systems typically require documents to be properly formatted and segmented, which is often done with relatively simple pre-processing steps, disregarding topical coherence of segments. Systems generally rely on representations of individual sentences or paragraphs, which may lack crucial context, or document-level representations, which are too long for meaningful search results. To address this is…
Towards Reliable Retrieval in RAG Systems for Large Legal Datasets
Technical University of Darmstadt（德意志联邦共和国达姆斯塔特技术大学）；University of Trento（特伦托大学）；University of Florence（佛罗伦萨大学）；University of Bologna（博洛尼亚大学）；European University Institute（欧洲大学研究所）
University of Information Technology（信息技术大学）；Vietnam National University, Ho Chi Minh City（胡志明市越南国家大学）；National Institute of Informatics（国立信息学研究所）
This study deals with the problem of information retrieval (IR) for Vietnamese legal texts. Despite being well researched in many languages, information retrieval has still not received much attention from the Vietnamese research community. This is especially true for the case of legal documents, which are hard to process. This study proposes a new approach for information retrieval for Vietnamese legal documents using sentence-transformer. Besides, various experiments are conducted to make comparisons between different transformer models, ranking scores, syllable-level, and word-level trainin…
Optimizing Legal Document Retrieval in Vietnamese with Semi-Hard Negative Mining
Faculty of Computer Science, University of Information Technology, Ho Chi Minh City, Vietnam（信息科技学院计算机科学系，胡志明市，越南）；Faculty of Information Science and Engineering, University of Information Technology, Ho Chi Minh City, Vietnam（信息科技学院信息科学与工程系，胡志明市，越南）；Laboratory of Multimedia Communications, University of Information Technology, Ho Chi Minh City, Vietnam（信息科技学院多媒体通信实验室，胡志明市，越南）；Vietnam National University, Ho Chi Minh City, Vietnam（越南国家大学，胡志明市，越南）
Legal practitioners often face a vast amount of documents. Lawyers, for instance, search for appropriate precedents favorable to their clients, while the number of legal precedents is ever-growing. Although legal search engines can assist finding individual target documents and narrowing down the number of candidates, retrieved information is often presented as unstructured text and users have to examine each document thoroughly which could lead to information overloading. This also makes their statistical analysis challenging. Here, we present an end-to-end information extraction (IE) system …
Japan Advanced Institute of Science and Technology（日本北陆先端科学技术大学院大学）
We introduce efficient deep learning-based methods for legal document processing including Legal Document Retrieval and Legal Question Answering tasks in the Automated Legal Question Answering Competition (ALQAC 2022). In this competition, we achieve 1\textsuperscript{st} place in the first task and 3\textsuperscript{rd} place in the second task. Our method is based on the XLM-RoBERTa model that is pre-trained from a large amount of unlabeled corpus before fine-tuning to the specific tasks. The experimental results showed that our method works well in legal retrieval information tasks with lim…
NeuralMind Inteligência Artificial（NeuralMind 人工智能公司）；Institute of Computing, University of Campinas（坎皮纳斯大学计算研究所）；School of Electrical and Computer Engineering, University of Campinas（坎皮纳斯大学电气与计算机工程学院）
A typical information extraction pipeline consists of token- or span-level classification models coupled with a series of pre- and post-processing scripts. In a production pipeline, requirements often change, with classes being added and removed, which leads to nontrivial modifications to the source code and the possible introduction of bugs. In this work, we evaluate sequence-to-sequence models as an alternative to token-level classification methods for information extraction of legal and registration documents. We finetune models that jointly extract the information and generate the output a…
Named entity recognition for Serbian legal documents: Design, methodology and dataset development
The Institute for Artificial Intelligence Research and Development of Serbia（塞尔维亚人工智能研究与发展研究所）；Faculty of Technical Sciences, University of Novi Sad（诺威萨大学技术科学学院）
Japan Advanced Institute of Science and Technology（日本先进科学技术研究院）；National Institute of Informatics（国立信息学研究所）
We present our method for tackling the legal case retrieval task of the Competition on Legal Information Extraction/Entailment 2019. Our approach is based on the idea that summarization is important for retrieval. On one hand, we adopt a summarization based model called encoded summarization which encodes a given document into continuous vector space which embeds the summary properties of the document. We utilize the resource of COLIEE 2018 on which we train the document representation model. On the other hand, we extract lexical features on different parts of a given query and its candidates.…
Arguments, counter-arguments, facts, and evidence obtained via documents related to previous court cases are of essential need for legal professionals. Therefore, the process of automatic information extraction from documents containing legal opinions related to court cases can be considered to be of significant importance. This study is focused on the identification of sentences in legal opinion texts which convey different perspectives on a certain topic or entity. We combined several approaches based on semantic analysis, open information extraction, and sentiment analysis to achieve our ob…
Computing similarity between two legal case documents is an important and challenging task in Legal IR, for which text-based and network-based measures have been proposed in literature. All prior network-based similarity methods considered a precedent citation network among case documents only (PCNet). However, this approach misses an important source of legal knowledge -- the hierarchy of legal statutes that are applicable in a given legal jurisdiction (e.g., country). We propose to augment the PCNet with the hierarchy of legal statutes, to form a heterogeneous network Hier-SPCNet, having cit…
How do complex social systems evolve in the modern world? This question lies at the heart of social physics, and network analysis has proven critical in providing answers to it. In recent years, network analysis has also been used to gain a quantitative understanding of law as a complex adaptive system, but most research has focused on legal documents of a single type, and there exists no unified framework for quantitative legal document analysis using network analytical tools. Against this background, we present a comprehensive framework for analyzing legal documents as multi-dimensional, dyn…
