作为税务律师的大语言模型：法律能力涌现的案例研究
Large Language Models as Tax Attorneys: A Case Study in Legal Capabilities Emergence
AI总结：
本研究以税法适用为场景开展案例研究，验证大语言模型的法律理解能力呈涌现性提升，结合少样本提示与正确法律文本可获高准确率，但尚未达专业税务律师水平，对法律行业和AI治理有重要意义。
摘要：
更好地理解大语言模型（LLMs）的法律分析能力，有助于提升法律服务效率、对人工智能进行治理，以及利用LLMs识别法律中的不一致之处。本文探究了LLMs在税法适用方面的能力。我们选择这一法律领域，是因为其结构允许我们搭建覆盖数千个示例的自动化验证流程，该领域既需要逻辑推理和数学能力，也让我们能以与公民和企业现实经济生活相关的方式测试LLM的能力。实验结果表明，LLM呈现出涌现性的法律理解能力，OpenAI每一代后续发布的模型性能都有所提升。我们通过检索和利用相关法律依据开展实验，评估向LLM提供额外法律背景信息的影响。研究还发现，少样本提示（即提供问答对示例）能显著提升最先进模型GPT-4的性能。研究结果表明，LLMs尤其是结合提示优化和正确法律文本时，能达到很高的准确率，但尚未达到专业税务律师的水平。随着LLMs持续发展，其自主进行法律推理的能力可能对法律行业和AI治理产生重大影响。
英文摘要：
Better understanding of Large Language Models' (LLMs) legal analysis abilities can contribute to improving the efficiency of legal services, governing artificial intelligence, and leveraging LLMs to identify inconsistencies in law. This paper explores LLM capabilities in applying tax law. We choose this area of law because it has a structure that allows us to set up automated validation pipelines across thousands of examples, requires logical reasoning and maths skills, and enables us to test LLM capabilities in a manner relevant to real-world economic lives of citizens and companies. Our experiments demonstrate emerging legal understanding capabilities, with improved performance in each subsequent OpenAI model release. We experiment with retrieving and utilising the relevant legal authority to assess the impact of providing additional legal context to LLMs. Few-shot prompting, presenting examples of question-answer pairs, is also found to significantly enhance the performance of the most advanced model, GPT-4. The findings indicate that LLMs, particularly when combined with prompting enhancements and the correct legal texts, can perform at high levels of accuracy but not yet at expert tax lawyer levels. As LLMs continue to advance, their ability to reason about law autonomously could have significant implications for the legal profession and AI governance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
