Large Language Models (LLMs) have shown remarkable performance across a wide range of downstream tasks. However, it is challenging for users to discern whether the responses of LLM are generated with certainty or are fabricated to meet user expectations. In this paper, we introduce Uncertainty Tripartite Testing Paradigm (Unc-TTP), a novel method for classifying LLM uncertainty by leveraging output inconsistency. Specifically, Unc-TTP performs three rounds of sampling under varying label injection interference, enumerating all possible outcomes, and uses the degree of output inconsistency as the indicator of the LLM's intrinsic uncertainty. To validate the effectiveness of this inconsistency-defined uncertainty, we draw inspiration from Active Learning, comparing the informativeness of actively selected in-context examples. Our experiments show that uncertainty examples selected via Unc-TTP are more informative than certainty examples. Furthermore, the Unc-TTP-guided uncertainty-based active example selection strategy outperforms existing methods, highlighting its effectiveness in classifying LLM uncertainty and enhancing in-context learning. This work not only underscores the potential of inconsistency-based uncertainty classification for both open- and closed-source LLMs but also presents a practical approach for leveraging uncertainty to improve LLM performance in real-world tasks.
PACIT: Unlocking the Power of Examples for Better In-Context Instruction Tuning
Nanjing University（南京大学）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Southern University of Science and Technology（南方科技大学）；Shanghai University of Finance and Economics（上海财经大学）
From Passive Metric to Active Signal: The Evolving Role of Uncertainty Quantification in Large Language Models
Salesforce AI Research（Salesforce AI研究院）；Intuit（Intuit公司）；Vanderbilt University（范德比大学）；University of California, Davis（加州大学戴维斯分校）；Vanderbilt University Medical Center（范德比大学医学中心）
Incorporating uncertainty quantification into travel mode choice modeling: a Bayesian neural network (BNN) approach and an uncertainty-guided active survey framework
Radboud University Nijmegen（拉德堡德大学奈梅亨分校）；Eindhoven University of Technology（埃因霍温理工大学）；KTH Royal Institute of Technology（瑞典皇家理工学院）；Ruhr-University Bochum（波鸿鲁尔大学）
DynActiveGS: Active Gaussian Splatting for Dynamic Scene Reconstruction
Tsinghua University, Shenzhen International Graduate School（清华大学深圳国际研究生院）；Huawei Technologies Ltd（华为技术有限公司）；Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））；Peng Cheng Laboratory（鹏城实验室）
Incremental Uncertainty-aware Performance Monitoring with Active Labeling Intervention
Siemens AG（西门子公司）；Goethe University Frankfurt（法兰克福歌德大学）；LMU Munich（慕尼黑大学）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；German Cancer Research Center (DKFZ)（德国癌症研究中心）；German Cancer Consortium (DKTK)（德国癌症联盟）
Uncertainty-Resilient Active Intention Recognition for Robotic Assistants
German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心（DFKI））；Osnabrück University（奥斯纳布吕克大学）；Joint Lab for AI & DS（人工智能与数据科学联合实验室）；Institute of Computer Science（计算机科学学院）
