The justice system has increasingly employed AI techniques to enhance efficiency, yet limitations remain in improving the quality of decision-making, particularly regarding transparency and explainability needed to uphold public trust in legal AI. To address these challenges, we propose a large language model based multi-agent framework named AgentsBench, which aims to simultaneously improve both efficiency and quality in judicial decision-making. Our approach leverages multiple LLM-driven agents that simulate the collaborative deliberation and decision making process of a judicial bench. We conducted experiments on legal judgment prediction task, and the results show that our framework outperforms existing LLM based methods in terms of performance and decision quality. By incorporating these elements, our framework reflects real-world judicial processes more closely, enhancing accuracy, fairness, and society consideration. AgentsBench provides a more nuanced and realistic methods of trustworthy AI decision-making, with strong potential for application across various case types and legal scenarios.
Toward Trustworthy Evaluation of Sustainability Rating Methodologies: A Human-AI Collaborative Framework for Benchmark Dataset Construction
Columbia University, USA(哥伦比亚大学);Case Western Reserve University, USA(凯斯西储大学);Hong Kong University of Science(香港科学大学);NVIDIA, USA(NVIDIA公司);Informatica Inc., USA(Informatica公司)
A Citation-Grounded Benchmark for Trustworthy Earnings Call Transcript Analysis with Large Language Models
American Express(美国运通);Singapore Decision Science Center of Excellence, American Express(美国运通新加坡决策科学卓越中心);Global Decision Science, American Express(美国运通全球决策科学部)
Probing Ethical Framework Representations in Large Language Models: Structure, Entanglement, and Methodological Challenges
School of Computer and Communication Sciences, École Polytechnique Fédérale de Lausanne(瑞士联邦理工学院计算机与通信科学学院);Digital Humanities Lab, École Polytechnique Fédérale de Lausanne(瑞士联邦理工学院数字人文实验室)
TrajAD: Trajectory Anomaly Detection for Trustworthy LLM Agents
School of Software, Shandong University(山东大学软件学院);Sonli Holding Group Co., Ltd.(山东利集团有限公司);Shandong Huazhi Talent Technology Co., Ltd.(山东华智人才科技有限公司);Information Technology Service Center of People’s Court(人民法院信息技术服务中心)
Rare Diseases, Common Dilemmas: LLMs Prioritize Equal Resource Distribution over Patient Benefit in Decision-Making
Harvard T.H. Chan School of Public Health(哈佛陈曾熙公共卫生学院);Harvard Medical School(哈佛医学院);Clalit Health Services(克拉利特医疗服务机构);Harvard College(哈佛学院);Harvard-MIT Program in Health Sciences and Technology(哈佛-麻省理工健康科学与技术项目)
Generalization or Memorization: Data Contamination and Trustworthy Evaluation for Large Language Models
Key Laboratory of High Confidence Software Technologies (Peking University)(高可信软件技术重点实验室(北京大学));School of Computer Science, Peking University(北京大学计算机学院);Beijing Institute of Control Engineering(北京控制工程研究所);China Academy of Space Technology(中国空间技术研究院)
CausalAbstain: Enhancing Multilingual LLMs with Causal Reasoning for Trustworthy Abstention
Department of Computer Science, Hong Kong Baptist University(香港 Baptist 大学计算机科学系);School of Mathematics and Statistics, The University of Melbourne(墨尔本大学数学与统计学学院);School of Computing and Information Systems, Singapore Management University(新加坡管理大学计算与信息系统学院)
Green Shielding: A User-Centric Approach Towards Trustworthy AI
University of California, Berkeley(加州大学伯克利分校);University of Melbourne(墨尔本大学);University of California, San Francisco(加州大学旧金山分校);University of Georgia(佐治亚大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校)
Rethinking All Evidence: Enhancing Trustworthy Retrieval-Augmented Generation via Conflict-Driven Summarization
University of Chinese Academy of Sciences(中国科学院大学);Chinese Academy of Sciences(中国科学院);National University of Defense Technology(国防科技大学);Chongqing University of Technology(重庆理工大学)
TAME: A Trustworthy Test-Time Evolution of Agent Memory with Systematic Benchmarking
East China Normal University(东华师范大学);Shanghai Innovation Institute(上海创新研究院);Shanghai Key Laboratory of Computer Software Evaluating and Testing(上海计算机软件评测与测试重点实验室);Huawei Noah’s Ark Lab(华为诺亚实验室)
TSHA: A Benchmark for Visual Language Models in Trustworthy Safety Hazard Assessment Scenarios
City University of Hong Kong(香港城市大学);East China Normal University(华东师范大学);University of Western Australia(西澳大学);Shanghai AI Laboratory(上海人工智能实验室);Shanghai Development Center of Computer Software Technology(上海计算机软件技术开发中心);Zhejiang Gongshang University(浙江工商大学)
A comprehensive and trustworthy benchmark of AI methods for change detection in Earth observation
Bias Variance Labs, d.o.o.(Bias Variance Labs有限责任公司);University of Ljubljana(卢布尔雅那大学);Jožef Stefan Institute(约热夫·斯泰凡研究所);Télécom Paris(巴黎电信学院);Institut Polytechnique de Paris(巴黎综合理工学院)
Towards trustworthy multi-modal motion prediction: Holistic evaluation and interpretability of outputs
University of Alcala(阿尔卡拉大学);AI Sweden(瑞典人工智能研究所);Zenseact AB(Zenseact公司);Linköping University(林雪平大学);Chalmers University of Technology(查尔姆斯理工大学);European Commission, Joint Research Centre(欧盟委员会联合研究中心)
Trustworthy Self-Composable Big-Data-as-a-Service: An LLM-Orchestrated Multi-Agent Framework for Automated Data Engineering, AutoML, MLOps Deployment, and Drift-Aware Lifecycle Optimization
School of Information, Computer, and Communication Technology(信息、计算机与通信技术学院);Sirindhorn International Institute of Technology, Thammasat University(素金国际技术研究所,泰国 Thammasat 大学)
SkinGPT-X: A Self-Evolving Collaborative Multi-Agent System for Transparent and Trustworthy Dermatological Diagnosis
School of Data Science, The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳)数据科学学院);Department of Dermatology, Tianjin Institute of Integrative Dermatology, Tianjin Academy of Traditional Chinese Medicine Affiliated Hospital(天津中医药大学附属医院天津市中西医结合皮肤病研究所皮肤科);Department of Dermatology, Beijing AnZhen Hospital, Capital Medical University(首都医科大学附属北京安贞医院皮肤科)