Rethinking the Uncertainty: A Critical Review and Analysis in the Era of Large Language Models
Virginia Tech(弗吉尼亚理工大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);The University of Texas at Dallas(德克萨斯大学达拉斯分校);University of California, Davis(加州大学戴维斯分校)
In recent years, Large Language Models (LLMs) have become fundamental to a broad spectrum of artificial intelligence applications. As the use of LLMs expands, precisely estimating the uncertainty in their predictions has become crucial. Current methods often struggle to accurately identify, measure, and address the true uncertainty, with many focusing primarily on estimating model confidence. This discrepancy is largely due to an incomplete understanding of where, when, and how uncertainties are injected into models. This paper introduces a comprehensive framework specifically designed to identify and understand the types and sources of uncertainty, aligned with the unique characteristics of LLMs. Our framework enhances the understanding of the diverse landscape of uncertainties by systematically categorizing and defining each type, establishing a solid foundation for developing targeted methods that can precisely quantify these uncertainties. We also provide a detailed introduction to key related concepts and examine the limitations of current methods in mission-critical and safety-sensitive applications. The paper concludes with a perspective on future directions aimed at enhancing the reliability and practical adoption of these methods in real-world scenarios.
Rethinking Prospect Theory for LLMs: Revealing the Instability of Decision-Making under Epistemic Uncertainty
Hong Kong University of Science and Technology(香港科技大学);Huazhong University of Science and Technology(华中科技大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校)
PERP: Rethinking the Prune-Retrain Paradigm in the Era of LLMs
Department for AI in Society, Science, and Technology, Zuse Institute Berlin, Germany(人工智能与社会、科学与技术部门,柏林Zuse研究所);Institute of Mathematics, Technische Universität Berlin, Germany(数学研究所,柏林技术大学)
Rethinking Agent Design: From Top-Down Workflows to Bottom-Up Skill Evolution
Centre for Frontier AI Research (CFAR), Agency for Science, Technology and Research (A*STAR)(前沿人工智能研究中心(CFAR)、科技研究局(A*STAR));Institute of High Performance Computing, Agency for Science, Technology and Research (A*STAR)(高性能计算研究所、科技研究局(A*STAR));National University of Singapore(国立新加坡大学);Tsinghua University(清华大学);University of Electronic Science and Technology of China(电子科技大学)
Beyond the Leaderboard: Rethinking Medical Benchmarks for Large Language Models
Stanford University(斯坦福大学);Shenzhen University(深圳大学);The Chinese University of Hong Kong(香港中文大学);City University of Hong Kong(香港城市大学);Renmin University of China(中国人民大学)
Rethinking Data Protection in the (Generative) Artificial Intelligence Era
The State Key Laboratory of Blockchain and Data Security(区块链与数据安全国家重点实验室);Nanyang Technological University(南洋理工大学);University of Maryland(马里兰大学);IBM Research(IBM研究院);CISPA Helmholtz Center for Information Security(CISPA 欧洲信息安全部分);University of Oxford(牛津大学)
Rethinking Human-AI Collaboration in Complex Medical Decision Making: A Case Study in Sepsis Diagnosis
Northeastern University(东北大学);Massachusetts Institute of Technology(麻省理工学院);The Ohio State University(俄亥俄州立大学);Rensselaer Polytechnic Institute(伦斯勒理工学院);The Ohio State University Wexner Medical Center(俄亥俄州立大学韦克斯纳医学中心)
Trustworthy Quantum Machine Learning: A Roadmap for Reliability, Robustness, and Security in the NISQ Era
Department of Electrical Engineering and Computer Science, University of Stavanger, Norway(斯瓦尔巴大学电气工程与计算机科学系);School of Physics, Engineering and Technology, University of York, UK(约克大学物理、工程与技术学院)