arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.26233cs.AIcs.CY
收藏

说服性与大语言模型作为法律决策工具

Assessing and Explaining the Persuadability of Large Language Models as Legal Decision Tools

School of Law;Criminology Maynooth University Maynooth Ireland;School of Computer Science University College Dublin Dublin Ireland;Criminology Maynooth University;School of Computer Science University College Dublin

Oisin Suttle, David Lillis

AI总结:

研究大语言模型在法律决策中的说服性,分析其对法律论点的响应机制及影响因素,探讨模型在法律和行政领域的应用可行性。

摘要:

随着大语言模型(LLMs)被提出作为法律决策助手,甚至第一审决策者,在各种司法和行政情境中,有必要探索它们如何回答法律问题,特别是导致它们以某种方式决定难题的因素。法律决策的一个特定特征是需要回应双方提出的论点。法律决策者必须能够参与并回应,包括可能被双方的论点说服,但不应过度被说服,受特别有说服力的辩护者影响,而应基于案件的实质。我们探讨前沿的开放式和封闭式权重LLMs如何响应法律论点,报告原始实验结果,考察提出论点的辩护者质量如何影响模型同意特定法律观点的可能性,并探索驱动这些结果的因素。我们的结果对在法律和行政领域采用LLMs的可行性具有影响。

英文摘要:

As Large Language Models (LLMs) are proposed as legal decision assistants, and even first-instance decision-makers, across a range of judicial and administrative contexts, it becomes essential to explore how they answer legal questions, and in particular the factors that lead them to decide difficult questions. A specific feature of legal decisions is the need to respond to arguments advanced by contending parties. A legal decision-maker must be able to engage with, and respond to, including through being potentially persuaded by, these arguments. Conversely, they should not be unduly persuadable, deciding cases based on the skills of the advocates rather than the merits of the case. In this paper we explore how frontier open- and closed-weights LLMs respond to legal arguments. We propose a metric to measure persuadability in the trilateral setting in which competing advocates seek to persuade a judge of opposite conclusions. We report original experimental results measuring how far the quality of the advocate making arguments affects the likelihood that a given model will agree with a particular legal point of view. We further examine how far models are capable of distinguishing between stronger and weaker arguments and how far model judgments in this domain are affected by positional bias. Through parallel bilateral trials we show how the trilateral setting changes the demands on judge models, and in turn their apparent persuadability. Finally we examine the specific features of arguments that affect persuasion, including the relative contribution of legal content and rhetorical form, the extent to which model persuasion tracks human expert judgments of argument quality, and the extent to which argument quantity, diversity and type affect persuasive outcomes. Our results have implications for the feasibility of adopting LLMs across legal and administrative settings.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑