In recent years, the utilization of large language models for natural language dialogue has gained momentum, leading to their widespread adoption across various domains. However, their universal competence in addressing challenges specific to specialized fields such as law remains a subject of scrutiny. The incorporation of legal ethics into the model has been overlooked by researchers. We asserts that rigorous ethic evaluation is essential to ensure the effective integration of large language models in legal domains, emphasizing the need to assess domain-specific proficiency and domain-specific ethic. To address this, we propose a novelty evaluation methodology, utilizing authentic legal cases to evaluate the fundamental language abilities, specialized legal knowledge and legal robustness of large language models (LLMs). The findings from our comprehensive evaluation contribute significantly to the academic discourse surrounding the suitability and performance of large language models in legal domains.
How Do LLMs Understand Legal Philosophy? Diagnosing Legal-Philosophical Reasoning in LLMs
The University of Osaka(大阪大学);UC Davis(加州大学戴维斯分校);UC Berkeley(加州大学伯克利分校);Peking University(北京大学);University of Wisconsin–Madison(威斯康星大学麦迪逊分校);Shanghai Jiao Tong University(上海交通大学);NII(信息学研究所(日本));City University of Hong Kong(香港城市大学)
When Large Language Models Meet Law: Dual-Lens Taxonomy, Technical Advances, and Ethical Governance
School of Law, China University of Political Science and Law(中国政法大学法学院);Zhejiang University of Finance and Economics Dongfang College(浙江财经大学东洋学院);School of Information Management for Law, China University of Political Science and Law(中国政法大学信息管理法学院);Department of Artificial Intelligence, Chung-Ang University(成均馆大学人工智能系);Department of Data Science and Artificial Intelligence, The Hong Kong Polytechnic University(香港理工大学数据科学与人工智能系)
Automating Legal Interpretation with LLMs: Retrieval, Generation, and Evaluation
Wangxuan Institute of Computer Technology, Peking University(北京大学王轩计算机技术研究所);Institute for Artificial Intelligence, Peking University(北京大学人工智能研究所);Peking University Law School(北京大学法学院)
Can LLMs Time Travel? Enhancing Temporal Consistency in Legal Agentic Search through Reinforcement Learning
Department of Computer Science and Engineering, HKUST, Hong Kong SAR, China(香港科技大学计算机科学与工程系);School of Law, Tsinghua University, Beijing, China(清华大学法学院);Cheriton School of Computer Science, University of Waterloo, Waterloo, Canada(滑铁卢大学丘成桐计算机科学系)
LawGPT: Knowledge-Guided Data Generation and Its Application to Legal LLM
National Key Laboratory for Novel Software Technology(现代软件工程国家重点实验室);School of Artificial Intelligence, Nanjing University(南京大学人工智能学院);School of Intelligence Science and Technology, Nanjing University(南京大学智能科学与技术学院)
Large Language Models Meet Legal Artificial Intelligence: A Survey
School of Computer Science and Engineering, Sun Yat-sen University(中山大学计算机科学与工程学院);China Mobile Internet Co., Ltd.(中国移动互联网有限公司);School of Computer Science, South China Normal University(华南师范大学计算机科学学院)
PLawBench: A Rubric-Based Benchmark for Evaluating LLMs in Real-World Legal Practice
Alibaba Group(阿里巴巴集团);Skylenage;Shanghai Jiao Tong University(上海交通大学);Tsinghua University(清华大学);University of Waterloo(滑铁卢大学);University of Pittsburgh(匹兹堡大学);University of Bologna(博洛尼亚大学)
Can LLMs Reason in a Legally Meaningful Manner? A Small-scale Study on European Court of Human Rights Cases
University of Copenhagen(哥本哈根大学);Faculty of Law, University of Copenhagen(哥本哈根大学法学院);Department of Computer Science, University of Copenhagen(哥本哈根大学计算机科学系)
Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models
Yale University(耶鲁大学);Yale Law School(耶鲁法学院);Stanford University(斯坦福大学);Stanford Law School(斯坦福法学院);Stanford Institute for Economic Policy Research(斯坦福经济政策研究所);Regulation, Evaluation, and Governance Lab(规制、评估与治理实验室)
Legal$Δ$: Enhancing Legal Reasoning in LLMs via Reinforcement Learning with Chain-of-Thought Guided Information Gain
School of Computer Science and Engineering, Northeastern University(东北大学计算机科学与工程学院);Department of Computer Science and Technology, Institute for AI, Tsinghua University(清华大学人工智能研究院计算机科学与技术系);Microsoft Research Asia(微软亚洲研究院)
ELLA: Empowering LLMs for Interpretable, Accurate and Informative Legal Advice
Wangxuan Institute of Computer Technology, Peking University(北京大学王选计算机研究所);School of Intelligence Science and Technology, Peking University(北京大学智能科学与技术学院);School of Electronics Engineering and Computer Science, Peking University(北京大学电子工程与计算机科学学院)