The Brazilian judiciary has a large workload, resulting in a long time to finish legal proceedings. Brazilian National Council of Justice has established in Resolution 469/2022 formal guidance for document and process digitalization opening up the possibility of using automatic techniques to help with everyday tasks in the legal field, particularly in a large number of texts yielded on the routine of law procedures. Notably, Artificial Intelligence (AI) techniques allow for processing and extracting useful information from textual data, potentially speeding up the process. However, datasets from the legal domain required by several AI techniques are scarce and difficult to obtain as they need labels from experts. To address this challenge, this article contributes with four datasets from the legal domain, two with documents and metadata but unlabeled, and another two labeled with a heuristic aiming at its use in textual semantic similarity tasks. Also, to evaluate the effectiveness of the proposed heuristic label process, this article presents a small ground truth dataset generated from domain expert annotations. The analysis of ground truth labels highlights that semantic analysis of domain text can be challenging even for domain experts. Also, the comparison between ground truth and heuristic labels shows that heuristic labels are useful.
Otto von Guericke University Magdeburg（马格德堡奥托·冯·格里克大学）
Getting an overview over the legal domain has become challenging, especially in a broad, international context. Legal question answering systems have the potential to alleviate this task by automatically retrieving relevant legal texts for a specific statement and checking whether the meaning of the statement can be inferred from the found documents. We investigate a combination of the BM25 scoring method of Elasticsearch with word embeddings trained on English translations of the German and Japanese civil law. For this, we define criteria which select a dynamic number of relevant documents ac…
Sense representations have gone beyond word representations like Word2Vec, GloVe and FastText and achieved innovative performance on a wide range of natural language processing tasks. Although very useful in many applications, the traditional approaches for generating word embeddings have a strict drawback: they produce a single vector representation for a given word ignoring the fact that ambiguous words can assume different meanings. In this paper, we explore unsupervised sense representations which, different from traditional word embeddings, are able to induce different senses of a word by…
VLSP 2023 -- LTER: A Summary of the Challenge on Legal Textual Entailment Recognition
Vietnam Judicial Academy（越南司法学院）；Hanoi Law University（河内法律大学）；Hung Yen University of Technology and Education（兴安工业大学）；Ho Chi Minh City University of Science（胡志明市科学大学）；Japan Advanced Institute of Science and Technology（日本先进科学技术学院）；The Institute of Statistical Mathematics（统计数学研究所）；National Institute of Informatics（情报学研究所）
University of Lisbon（里斯本大学）；Instituto Superior Técnico（高等技术学院）；University of Algarve（阿尔加维大学）；Faculty of Science and Technology, University of Algarve（阿尔加维大学科学与技术学院）；CCMAR（海洋科学研究中心）；INESC-ID（里斯本系统与计算机工程、技术与科学研究所）
Being able to clearly interpret legal texts and fully understanding our rights, obligations and other legal norms has become progressively more important in the digital society. However, simply giving citizens access to the laws is not enough, as there is a need to provide meaningful information that cater to their specific queries and needs. For this, it is necessary to extract the relevant semantic information present in legal texts. Thus, we introduce the SNR (Semantic Norm Recognition) system, an automatic semantic information extraction system trained on a domain-specific (legal) text cor…
Black-Box Analysis: GPTs Across Time in Legal Textual Entailment Task
National Institute of Informatics（国立信息学研究所）；University of Alberta（阿尔伯塔大学）；Alberta Machine Intelligence Institute（阿尔伯塔机器智能研究所）；Imperial College London（伦敦帝国理工学院）；Royal Holloway, University of London（伦敦大学皇家霍洛威学院）
This work investigates legal concepts and their expression in Portuguese, concentrating on the "Order of Attorneys of Brazil" Bar exam. Using a corpus formed by a collection of multiple-choice questions, three norms related to the Ethics part of the OAB exam, language resources (Princeton WordNet and OpenWordNet-PT) and tools (AntConc and Freeling), we began to investigate the concepts and words missing from our repertory of concepts and words in Portuguese, the knowledge base OpenWordNet-PT. We add these concepts and words to OpenWordNet-PT and hence obtain a representation of these texts tha…
Federal University of Rio Grande do Sul（南里奥格兰德州联邦大学）
BERT (Bidirectional Encoder Representations from Transformers) and ALBERT (A Lite BERT) are methods for pre-training language models which can later be fine-tuned for a variety of Natural Language Understanding tasks. These methods have been applied to a number of such tasks (mostly in English), achieving results that outperform the state-of-the-art. In this paper, our contribution is twofold. First, we make available our trained BERT and Albert model for Portuguese. Second, we compare our monolingual and the standard multilingual models using experiments in semantic textual similarity, recogn…
This paper introduces a novel approach to tackle the existing gap on message translations in dialogue systems. Currently, submitted messages to the dialogue systems are considered as isolated sentences. Thus, missing context information impede the disambiguation of homographs words in ambiguous sentences. Our approach solves this disambiguation problem by using concepts over existing ontologies.
Universidade de São Paulo（圣保罗大学）；Política por Inteiro
Even though most of its energy generation comes from renewable sources, Brazil is one of the largest emitters of greenhouse gases in the world, due to intense farming and deforestation of biomes such as the Amazon Rainforest, whose preservation is essential for compliance with the Paris Agreement. Still, regardless of lobbies or prevailing political orientation, all government legal actions are published daily in the Brazilian Federal Official Gazette (BFOG, or "Diário Oficial da União" in Portuguese). However, with hundreds of decrees issued every day by the authorities, it is absolutely burd…
The success of pre-trained word embeddings has motivated its use in tasks in the biomedical domain. The BERT language model has shown remarkable results on standard performance metrics in tasks such as Named Entity Recognition (NER) and Semantic Textual Similarity (STS), which has brought significant progress in the field of NLP. However, it is unclear whether these systems work seemingly well in critical domains, such as legal or medical. For that reason, in this work, we propose an adversarial evaluation scheme on two well-known datasets for medical NER and STS. We propose two types of attac…
In this paper, we present the methodology and the results obtained by our teams, dubbed Blue Man Group, in the ASSIN (from the Portuguese {\it Avaliação de Similaridade Semântica e Inferência Textual}) competition, held at PROPOR 2016\footnote{International Conference on the Computational Processing of the Portuguese Language - http://propor2016.di.fc.ul.pt/}. Our team's strategy consisted of evaluating methods based on semantic word vectors, following two distinct directions: 1) to make use of low-dimensional, compact, feature sets, and 2) deep learning-based strategies dealing with high-dime…
Text normalization techniques based on rules, lexicons or supervised training requiring large corpora are not scalable nor domain interchangeable, and this makes them unsuitable for normalizing user-generated content (UGC). Current tools available for Brazilian Portuguese make use of such techniques. In this work we propose a technique based on distributed representation of words (or word embeddings). It generates continuous numeric vectors of high-dimensionality to represent words. The vectors explicitly encode many linguistic regularities and patterns, as well as syntactic and semantic word …
Even in highly-developed countries, as many as 15-30\% of the population can only understand texts written using a basic vocabulary. Their understanding of everyday texts is limited, which prevents them from taking an active role in society and making informed decisions regarding healthcare, legal representation, or democratic choice. Lexical simplification is a natural language processing task that aims to make text understandable to everyone by replacing complex vocabulary and expressions with simpler ones, while preserving the original meaning. It has attracted considerable attention in the…
Indian Institute of Technology, Kharagpur（印度理工学院卡拉格普尔分校）
The task of Legal Statute Identification (LSI) aims to identify the legal statutes that are relevant to a given description of Facts or evidence of a legal case. Existing methods only utilize the textual content of Facts and legal articles to guide such a task. However, the citation network among case documents and legal statutes is a rich source of additional information, which is not considered by existing models. In this work, we take the first step towards utilising both the text and the legal citation network for the LSI task. We curate a large novel dataset for this task, including Facts…
Indian Institute of Technology Kharagpur（印度理工学院克勒格布尔分校）；Tata Research Development and Design Centre (TRDDC)（塔塔研究开发与设计中心（TRDDC））；Data61, CSIRO（澳大利亚联邦科学与工业研究组织 Data61）；Cyber Security CRC（网络安全合作研究中心）
Computing similarity between two legal documents is an important and challenging task in the domain of Legal Information Retrieval. Finding similar legal documents has many applications in downstream tasks, including prior-case retrieval, recommendation of legal articles, and so on. Prior works have proposed two broad ways of measuring similarity between legal documents - analyzing the precedent citation network, and measuring similarity based on textual content similarity measures. But there has not been a comprehensive comparison of these existing methods on a common platform. In this paper,…
The Brazilian Supreme Court receives tens of thousands of cases each semester. Court employees spend thousands of hours to execute the initial analysis and classification of those cases -- which takes effort away from posterior, more complex stages of the case management workflow. In this paper, we explore multimodal classification of documents from Brazil's Supreme Court. We train and evaluate our methods on a novel multimodal dataset of 6,510 lawsuits (339,478 pages) with manual annotation assigning each page to one of six classes. Each lawsuit is an ordered sequence of pages, which are stor…
COPPE/Universidade Federal do Rio de Janeiro (UFRJ)（里约热内卢联邦大学科佩研究所）
Over the last few decades, institutions around the world have been challenged to deal with the sheer volume of information captured in unstructured formats, especially in textual documents. The so called Digital Transformation age, characterized by important technological advances and the advent of disruptive methods in Artificial Intelligence, offers opportunities to make better use of this information. Recent techniques in Natural Language Processing (NLP) with Deep Learning approaches allow to efficiently process a large volume of data in order to obtain relevant information, to identify pa…
GRAF: Graph Retrieval Augmented by Facts for Romanian Legal Multi-Choice Question Answering
National University of Science and Technology POLITEHNICA Bucharest, Faculty of Automatic Control and Computers（波兰技术大学布加勒斯特分校国家科学与技术大学，自动控制与计算机学院）；Technical University of Munich（慕尼黑技术大学）；Paris 1 Panthéon-Sorbonne University（巴黎第一大学）；University of Bucharest（布加勒斯特大学）
NOWJ@COLIEE 2025: A Multi-stage Framework Integrating Embedding Models and Large Language Models for Legal Retrieval and Entailment
VNU University of Engineering and Technology（越南工程大学）；Japan Advanced Institute of Science and Technology（日本先进科学研究院）；Center for Juris-Informatics, ROIS-DS（法律信息中心，ROIS-DS）；Research and Development Center for Large Language Models, NII（大型语言模型研发中心，NII）
Named entity recognition for Serbian legal documents: Design, methodology and dataset development
The Institute for Artificial Intelligence Research and Development of Serbia（塞尔维亚人工智能研究与发展研究所）；Faculty of Technical Sciences, University of Novi Sad（诺威萨大学技术科学学院）
Artificial intelligence is being utilized in many domains as of late, and the legal system is no exception. However, as it stands now, the number of well-annotated datasets pertaining to legal documents from the Supreme Court of the United States (SCOTUS) is very limited for public use. Even though the Supreme Court rulings are public domain knowledge, trying to do meaningful work with them becomes a much greater task due to the need to manually gather and process that data from scratch each time. Hence, our goal is to create a high-quality dataset of SCOTUS court cases so that they may be rea…
CORE: A Few-Shot Company Relation Classification Dataset for Robust Domain Adaptation
IESEG School of Management（IESEG管理学院）；Univ. Lille（里尔大学）；CNRS（法国国家科学研究中心）；Research Centre for Information Systems Engineering, KU Leuven（鲁汶大学信息系统工程研究中心）；KU Leuven（鲁汶大学）
Identification of Regulatory Requirements Relevant to Business Processes: A Comparative Study on Generative AI, Embedding-based Ranking, Crowd and Expert-driven Methods
Technical University of Munich（慕尼黑工业大学）；University of Queensland（昆士兰大学）
University of Washington（华盛顿大学）；Allen Institute for AI（艾伦人工智能研究所）；Meta AI
We present Branch-Train-Merge (BTM), a communication-efficient algorithm for embarrassingly parallel training of large language models (LLMs). We show it is possible to independently train subparts of a new class of LLMs on different subsets of the data, eliminating the massive multi-node synchronization currently required to train LLMs. BTM learns a set of independent expert LMs (ELMs), each specialized to a different textual domain, such as scientific or legal text. These ELMs can be added and removed to update data coverage, ensembled to generalize to new domains, or averaged to collapse ba…
