In a legal system, judgment consistency is regarded as one of the most important manifestations of fairness. However, due to the complexity of factual elements that impact sentencing in real-world scenarios, few works have been done on quantitatively measuring judgment consistency towards real-world data. In this paper, we propose an evaluation metric for judgment inconsistency, Legal Inconsistency Coefficient (LInCo), which aims to evaluate inconsistency between data groups divided by specific features (e.g., gender, region, race). We propose to simulate judges from different groups with legal judgment prediction (LJP) models and measure the judicial inconsistency with the disagreement of the judgment results given by LJP models trained on different groups. Experimental results on the synthetic data verify the effectiveness of LInCo. We further employ LInCo to explore the inconsistency in real cases and come to the following observations: (1) Both regional and gender inconsistency exist in the legal system, but gender inconsistency is much less than regional inconsistency; (2) The level of regional inconsistency varies little across different time periods; (3) In general, judicial inconsistency is negatively correlated with the severity of the criminal charges. Besides, we use LInCo to evaluate the performance of several de-bias methods, such as adversarial learning, and find that these mechanisms can effectively help LJP models to avoid suffering from data bias.
Legal Judgment Reimagined: PredEx and the Rise of Intelligent AI Interpretation in Indian Courts
Symbiosis Law School(symbiosis 法学院);IISER Kolkata(印度科学教育与研究学院加尔各答分校);Indian Institute of Technology Kanpur(坎普尔印度理工学院);Manipal University Jaipur(斋浦尔马尼帕尔大学)
NyayaMind- A Framework for Transparent Legal Reasoning and Judgment Prediction in the Indian Legal System
IIT Kanpur, India(印度理工学院坎普尔分校);IIT Kharagpur, India(印度理工学院克勒格布尔分校);Symbiosis Law School Pune, India(印度共生法学院浦那分校);Dattam Labs, India(印度Dattam实验室);University of Birmingham, Dubai, United Arab Emirates(英国伯明翰大学迪拜校区)
Legal Syllogism Prompting: Teaching Large Language Models for Legal Judgment Prediction
Peking University Law School(北京大学法学院);Institute for Artificial Intelligence, Peking University(北京大学人工智能研究院);PKU-WUHAN Institute for Artificial Intelligence(北大-武汉人工智能研究院)
AppellateGen: A Benchmark for Appellate Legal Judgment Generation
Nanyang Technological University(南洋理工大学);Ocean University of China(中国海洋大学);The Hong Kong Polytechnic University(香港理工大学);Hong Kong University of Science and Technology(香港科技大学);Tsinghua University(清华大学);University of Science and Technology of China(中国科学技术大学);Cornell University(康奈尔大学)
NyayaRAG: Realistic Legal Judgment Prediction with RAG under the Indian Common Law System
Indian Institute of Technology Kanpur(印度理工学院坎浦尔分校);SRM Institute of Science and Technology(SRM科学与技术学院);Symbiosis Law School Pune(苏比斯法学院浦那分校);IISER Kolkata(印度科学研究所库勒卡布尔分校);University of Birmingham, Dubai, United Arab Emirates(布里斯托尔大学迪拜分校)
AR-BENCH: Benchmarking Legal Reasoning with Judgment Error Detection, Classification and Correction
SKLCCSE Beihang University(北京航空航天大学软件学院);SCCE University of Science and Technology Beijing(北京科技大学计算机学院);Sino-French Engineer School Beihang University(北京航空航天大学中法工程师学院);People’s Procuratorate of Beijing Municipality(北京市人民检察院)
ILDC for CJPE: Indian Legal Documents Corpus for Court Judgment Prediction and Explanation
Indian Institute of Technology Kanpur(坎普尔印度理工学院);Indian Institute of Science Education and Research Kolkata(加尔各答印度科学教育与研究学院);West Bengal National University of Juridical Sciences(西孟加拉国立法科大学)
CAIL2018: A Large-Scale Legal Dataset for Judgment Prediction
Tsinghua University(清华大学);Peking University(北京大学);Chinese Academy of Sciences(中国科学院);China Justice Big Data Institute(中国司法大数据研究院);Supreme People’s Court(最高人民法院)
Corpus for Automatic Structuring of Legal Documents
EkStep Foundation(EkStep基金会);Thoughtworks Technologies India Pvt Ltd.(印度思特沃克技术私人有限公司);Agami;Indian Institute of Technology Kanpur (IIT-K)(印度理工学院坎普尔分校(IIT-K))
SEMDR: A Semantic-Aware Dual Encoder Model for Legal Judgment Prediction with Legal Clue Tracing
College of Science, Northeastern University(东北大学理学院);School of Computer Science and Engineering, Southern University of Science and Technology(南方科技大学计算机科学与工程学院);Department of Computer Science, University of Reading(雷丁大学计算机科学系)
Exploiting Contrastive Learning and Numerical Evidence for Confusing Legal Judgment Prediction
Zhejiang University(浙江大学);Alibaba Group(阿里巴巴集团);University of Massachusetts at Amherst(马萨诸塞大学阿默斯特分校);Nanyang Technological University(南洋理工大学);Shanghai AI Laboratory(上海人工智能实验室);Shanghai Institute for Advanced Study of Zhejiang University(浙江大学上海高等研究院)
Capturing Legal Reasoning Paths from Facts to Law in Court Judgments using Knowledge Graphs
Graduate School of Information Science and Technology, The University of Tokyo(东京大学信息科学与技术研究生院);The Canon Institute for Global Studies(Canon全球研究机构);Graduate Schools for Law and Politics, The University of Tokyo(东京大学法学与政治学研究生院);TKC Corporation(TKC公司)
From Judgments to Issues: Structured Extraction of Legal Reasoning with Citation-Hallucination Control
Quantitative and Digital Law Laboratory(量化与数字法律实验室);CIRSFID - Alma AI, Faculty of Law University of Bologna(CIRSFID - 阿尔玛AI,博洛尼亚大学法学院);Robert Schuman Centre European University Institute(罗伯特·施曼中心欧洲大学研究所)
JPO: Juris Policy Optimization for Structured Legal Reasoning in Criminal Judgment Prediction
Tencent(腾讯);Peking University(北京大学);Tsinghua University(清华大学);City University of Hong Kong(香港城市大学);University of California(加利福尼亚大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Zhejiang University(浙江大学);University of Hong Kong(香港大学)