SAILER：用于法律案件检索的结构感知预训练语言模型
SAILER: Structure-aware Pre-trained Language Model for Legal Case Retrieval
AI总结：
本文提出结构感知预训练语言模型 SAILER，通过利用法律文书结构、关注关键法律要素并融合多预训练目标，在无法律标注情况下提升法律案件检索性能。
摘要：
法律案件检索旨在为查询案件找到相关案件，在智能法律系统中发挥着核心作用。尽管预训练在即席检索任务中取得了成功，但面向法律案件检索的有效预训练策略仍有待探索。与一般文档相比，法律案件文档通常是具有内在逻辑结构的长文本序列。然而，大多数现有语言模型难以理解不同结构之间的长距离依赖关系。此外，与一般检索不同，法律领域中的相关性对关键法律要素十分敏感。即使关键法律要素中存在细微差异，也会显著影响对相关性的判断。然而，现有的面向通用目的设计的预训练语言模型尚不具备处理法律要素的能力。为解决这些问题，本文提出 SAILER，这是一种新的用于法律案件检索的结构感知预训练语言模型（Structure-Aware pre-traIned language model for LEgal case Retrieval）。其突出体现在以下三个方面：（1）SAILER 充分利用法律案件文档中包含的结构信息，并更加关注关键法律要素，类似于法律专家浏览法律案件文档的方式。（2）SAILER 采用非对称编码器—解码器架构来整合若干不同的预训练目标。通过这种方式，跨任务的丰富语义信息被编码到稠密向量中。（3）即使没有任何法律标注数据，SAILER 也具有强大的判别能力。它能够准确区分具有不同指控的法律案件。在公开可用的法律基准上进行的大量实验表明，我们的方法在法律案件检索中能够显著优于以往的最先进方法。
英文摘要：
Legal case retrieval, which aims to find relevant cases for a query case, plays a core role in the intelligent legal system. Despite the success that pre-training has achieved in ad-hoc retrieval tasks, effective pre-training strategies for legal case retrieval remain to be explored. Compared with general documents, legal case documents are typically long text sequences with intrinsic logical structures. However, most existing language models have difficulty understanding the long-distance dependencies between different structures. Moreover, in contrast to the general retrieval, the relevance in the legal domain is sensitive to key legal elements. Even subtle differences in key legal elements can significantly affect the judgement of relevance. However, existing pre-trained language models designed for general purposes have not been equipped to handle legal elements. To address these issues, in this paper, we propose SAILER, a new Structure-Aware pre-traIned language model for LEgal case Retrieval. It is highlighted in the following three aspects: (1) SAILER fully utilizes the structural information contained in legal case documents and pays more attention to key legal elements, similar to how legal experts browse legal case documents. (2) SAILER employs an asymmetric encoder-decoder architecture to integrate several different pre-training objectives. In this way, rich semantic information across tasks is encoded into dense vectors. (3) SAILER has powerful discriminative ability, even without any legal annotation data. It can distinguish legal cases with different charges accurately. Extensive experiments over publicly available legal benchmarks demonstrate that our approach can significantly outperform previous state-of-the-art methods in legal case retrieval.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
