Recently, Knowledge Graphs (KGs) have been successfully coupled with Large Language Models (LLMs) to mitigate their hallucinations and enhance their reasoning capability, such as in KG-based retrieval-augmented frameworks. However, current KG-LLM frameworks lack rigorous uncertainty estimation, limiting their reliable deployment in high-stakes applications. Directly incorporating uncertainty quantification into KG-LLM frameworks presents challenges due to their complex architectures and the intricate interactions between the knowledge graph and language model components. To address this gap, we propose a new trustworthy KG-LLM framework, Uncertainty Aware Knowledge-Graph Reasoning (UAG), which incorporates uncertainty quantification into the KG-LLM framework. We design an uncertainty-aware multi-step reasoning framework that leverages conformal prediction to provide a theoretical guarantee on the prediction set. To manage the error rate of the multi-step process, we additionally introduce an error rate control module to adjust the error rate within the individual components. Extensive experiments show that our proposed UAG can achieve any pre-defined coverage rate while reducing the prediction set/interval size by 40% on average over the baselines.
ReARTeR: Retrieval-Augmented Reasoning with Trustworthy Process Rewarding
Kuaishou Technology Co., Ltd.（快手科技有限公司）；Institute for Clarity in Documentation（文献清晰度研究所）；Inria Paris-Rocquencourt（法国国家信息与自动化研究所巴黎-罗康分所）；Rajiv Gandhi University（拉吉夫甘地大学）；Tsinghua University（清华大学）；Palmer Research Laboratories（帕尔默研究实验室）
CausalAbstain: Enhancing Multilingual LLMs with Causal Reasoning for Trustworthy Abstention
Department of Computer Science, Hong Kong Baptist University（香港 Baptist 大学计算机科学系）；School of Mathematics and Statistics, The University of Melbourne（墨尔本大学数学与统计学学院）；School of Computing and Information Systems, Singapore Management University（新加坡管理大学计算与信息系统学院）
TT-XAI: Trustworthy Clinical Text Explanations via Keyword Distillation and LLM Reasoning
Faculty of Computer and Information Science, University of Ljubljana, Slovenia（卢布尔雅那大学计算机与信息科学学院）；ICAM - Advanced Environmental Research Institute, West University of Timisoara, Romania（蒂米șoara西大学先进环境研究所）；Department of Computer Science, West University of Timisoara, Romania（蒂米șoa拉西大学计算机科学系）
TrustGeoGen: Formal-Verified Data Engine for Trustworthy Multi-modal Geometric Problem Solving
College of Computer Science and Artificial Intelligence, Fudan University（复旦大学计算机科学与人工智能学院）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；School of Artificial Intelligence, Shanghai Jiao Tong University（上海交通大学人工智能学院）
Deliberation on Priors: Trustworthy Reasoning of Large Language Models on Knowledge Graphs
MOE KLINNS Lab, Xi’an Jiaotong University（MOE KLINNS实验室，西安交通大学）；School of Computer Science and Technology, Xi’an Jiaotong University（计算机科学与技术学院，西安交通大学）；Shaanxi Province Key Laboratory of Big Data Knowledge Engineering（陕西省大数据知识工程重点实验室）；School of Artificial Intelligence, Chongqing University of Post and Telecommunications（人工智能学院，重庆邮电大学）；School of Computer Science, Northwestern Polytechnical University（计算机学院，西北工业大学）
Towards Trustworthy Retrieval Augmented Generation for Large Language Models: A Survey
Vanderbilt University（范德堡大学）；University of Notre Dame（圣母大学）；University of Oregon（俄勒冈大学）；Meta；Oracle Health AI（甲骨文健康人工智能公司）；Adobe Research（奥多比研究院）；Cisco AI Research（思科人工智能研究院）；North Carolina State University（北卡罗来纳州立大学）；The Hong Kong Polytechnic University（香港理工大学）；Air Force Research Lab（空军研究实验室）
Med-R$^2$: Crafting Trustworthy LLM Physicians via Retrieval and Reasoning of Evidence-Based Medicine
Center for Data Science, AAIS Peking University Beijing China（数据科学中心，北京大学北京中国）；Baichuan Inc. Beijing China（北川公司北京中国）；School of Computer Science Peking University Beijing China（计算机科学学院，北京大学北京中国）；Peking University（北京大学）；Baichuan Inc.（北川公司）
Knowledge-Augmented Multimodal Clinical Rationale Generation for Disease Diagnosis with Small Language Models
Hong Kong Baptist University（香港 Baptist 大学）；Shanxi University（山西大学）；Shanghai Institute for Advanced Study of Zhejiang University（浙江大学上海先进研究院）；Hong Kong University of Science and Technology（香港科技大学）；The University of Manchester（曼彻斯特大学）
DocScope: Benchmarking Verifiable Reasoning for Trustworthy Long-Document Understanding
School of Computer Science, National Engineering Research Center for Multimedia Software and Hubei Key Laboratory of Multimedia and Network Communication Engineering, Wuhan University, China（计算机学院，国家多媒体软件工程技术研究中心和湖北多媒体与网络通信工程重点实验室，武汉大学，中国）；Alibaba Group, Hangzhou, China（阿里巴巴集团，杭州，中国）；Independent Researcher（独立研究者）；Department of Machine Learning, Mohamed bin Zayed University of Artificial Intelligence, United Arab Emirates（机器学习系，Mohamed bin Zayed人工智能大学，阿拉伯联合酋长国）
SAFEFLOW: A Principled Protocol for Trustworthy and Transactional Autonomous Agent Systems
Texas A&M University（德州农工大学）；UC San Diego（加州大学圣地亚哥分校）；UC Irvine（加州大学欧文分校）；University of Wisconsin–Madison（威斯康星大学麦迪逊分校）；Carnegie Mellon University（卡内基梅隆大学）；University of Michigan（密歇根大学）；Columbia University（哥伦比亚大学）；Meta
NeuroSymbolic AI for Legal AI-TRISM: Trustworthy, Reliable, Interpretable, Safe Models
Department of Computer Science, AI Institute, University of South Carolina（南卡罗来纳大学计算机科学系，人工智能研究所）；Department of Computer Science and Electrical Engineering, University of Maryland, Baltimore County（马里兰大学巴尔的摩县分校计算机科学与电气工程系）；Department of Computer Science and Engineering, The Ohio State University（俄亥俄州立大学计算机科学与工程系）
An Explicit Syllogistic Legal Reasoning Framework for Large Language Models
Gaoling School of Artificial Intelligence（人工智能学院）；Renmin University of China（中国人民大学）；School of Information Technology and Management（信息科技与管理学院）；University of International Business and Economics（国际经济贸易大学）
The Fusion of Large Language Models and Formal Methods for Trustworthy AI Agents: A Roadmap
National University of Singapore（新加坡国立大学）；Peking University（北京大学）；Huazhong University of Science and Technology（华中科技大学）；Griffith University（格里菲斯大学）；Beijing Institute of Technology（北京理工大学）；Singapore Management University（新加坡管理大学）；University of Auckland（奥克兰大学）
Knowledge Graph Reasoning with Logics and Embeddings: Survey and Perspective
Zhejiang University（浙江大学）；University of Oxford（牛津大学）；The University of Edinburgh（爱丁堡大学）；Alibaba-Zhejiang University Joint Research Institute of Frontier Technologies（阿里巴巴-浙江大学前沿技术联合研究院）；Hangzhou Innovation Center, Zhejiang University（浙江大学杭州国际科创中心）
