arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.07871cs.HCcs.CY
收藏

异议无效!普通人能区分大语言模型与律师,但仍更青睐大语言模型的建议

Objection Overruled! Lay People can Distinguish Large Language Models from Lawyers, but still Favour Advice from an LLM

Eike Schneiders, Tina Seabrooke, Joshua Krook, Richard Hyde, Natalie Leesakul, Jeremie Clos, Joel Fischer

AI总结:

该研究通过三项共288人参与的实验发现,普通人虽能以高于随机的水平区分LLM与律师生成的法律建议,但在来源未知时更愿意执行LLM给出的法律建议,揭示了LLM在法律建议领域的受众偏好现象。

摘要:

大语言模型(LLM)似乎正渗透到各个领域,法律领域也不例外。本文呈现了三项实验(总样本量N=288)的结果,探究普通人对LLM生成的法律建议与律师生成的法律建议的执行意愿,以及区分二者的能力。实验1中,参与者在建议来源已知或未知的两种情况下,评判自己执行该法律建议的意愿。当建议来源未知时,参与者表示自己执行LLM生成建议的意愿显著更高。实验2复现了来源未知条件下的这一结果。有趣的是,尽管实验1和2中参与者表示更愿意执行LLM生成的建议,但在实验3中,参与者区分LLM生成文本与律师生成文本的表现显著高于随机水平。最后,我们讨论了研究发现的潜在解释与风险、研究局限性以及未来工作方向。

英文摘要:

Large Language Models (LLMs) are seemingly infiltrating every domain, and the legal context is no exception. In this paper, we present the results of three experiments (total N = 288) that investigated lay people's willingness to act upon, and their ability to discriminate between, LLM- and lawyer-generated legal advice. In Experiment 1, participants judged their willingness to act on legal advice when the source of the advice was either known or unknown. When the advice source was unknown, participants indicated that they were significantly more willing to act on the LLM-generated advice. The result of the source unknown condition was replicated in Experiment 2. Intriguingly, despite participants indicating higher willingness to act on LLM-generated advice in Experiments 1 and 2, participants discriminated between the LLM- and lawyer-generated texts significantly above chance-level in Experiment 3. Lastly, we discuss potential explanations and risks of our findings, limitations and future work.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑