通过知识图谱引导的多任务预训练增强预训练语言模型的法律推理能力
Enhancing Legal Reasoning in Pre-trained Language Models via Knowledge Graph-Guided Multi-Task Pre-training
AI总结：
针对预训练语言模型法律推理能力不足的问题，提出结合NSP、CLU、CLS目标的KG引导多任务预训练方法，在StVO等任务上取得显著性能提升且具备跨领域泛化能力。
摘要：
法律语言理解需要对半结构化的法规文档进行推理，但标准预训练目标并未明确针对这种结构化推理能力。当前法律语言模型仅依赖掩码语言建模，将法规文本视为扁平的 token 序列，忽略了潜在的层级约束。为弥合这一差距，我们提出了针对德国交通法规（StVO）的多任务预训练方法，该方法结合了三个互补目标：下一句预测（NSP）、知识图谱（KG）引导聚类（CLU）以及知识图谱（KG）感知段落分类（CLS）。此方法将结构化法律知识直接编码到模型权重中，使模型在推理时具备零样本法律推理能力，无需进行离散图遍历。我们在9个基于Transformer的编码器上开展了324次实验，在合成StVO问答任务中，我们的模型相较于基线取得了10.3%至155.0%的一致相对提升；与已确立的LexGLUE基线（Chalkidis等人，2022）相比，我们的模型在7个法律基准中的4个上优于先前结果，且在奥地利驾驶法规上实现了3.4%至15.8%的一致零样本迁移增益。这些实验结果表明，我们的训练方法可成功泛化至初始训练法律领域之外。
英文摘要：
Legal language understanding requires reasoning over semi-structured regulatory documents, yet standard pre-training objectives do not explicitly target this structural capability. Because current legal language models rely solely on masked language modeling, they treat statutory text as flat token sequences and ignore the underlying hierarchical constraints. To bridge this gap, we present a multi-task pre-training approach for German traffic law (StVO). Our approach combines three complementary objectives: Next Sentence Prediction (NSP), Knowledge Graph (KG) Guided Clustering (CLU), and Knowledge Graph (KG) Aware Paragraph Classification (CLS). This approach explicitly encodes structured legal knowledge directly into the model weights. This encoding enables zero-shot legal reasoning at inference time and removes the need for discrete graph traversal. We evaluate our approach across nine Transformer-based encoders and 324 experimental runs. On synthetic StVO question answering, our models demonstrate consistent relative improvements between 10.3% and 155.0% over baselines. Furthermore, relative to the established LexGLUE baselines (Chalkidis et al., 2022), our models improve over prior results on four of seven legal benchmarks. It also achieves consistent zero-shot transfer gains between 3.4% and 15.8% on Austrian driving regulations. These experimental results indicate that our training approach generalizes successfully beyond the initial training legal domain.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
