Large language models (LLMs) have demonstrated remarkable capabilities across diverse tasks. However, the truthfulness of their outputs is not guaranteed, and their tendency toward overconfidence further limits reliability. Uncertainty quantification offers a promising way to identify potentially unreliable outputs, but most existing methods rely on repeated sampling or auxiliary models, introducing substantial computational overhead. To address these limitations, we propose Semantic Token Clustering (STC), an efficient uncertainty quantification method that leverages the semantic information inherently encoded in LLMs. Specifically, we group tokens into semantically consistent clusters using embedding clustering and prefix matching, and quantify uncertainty based on the probability mass aggregated over the corresponding semantic cluster. Our approach requires only a single generation and does not depend on auxiliary models. Experimental results show that STC achieves performance comparable to state-of-the-art baselines while substantially reducing computational overhead.
Fact-Checking the Output of Large Language Models via Token-Level Uncertainty Quantification
Center for Artificial Intelligence Technology（人工智能技术中心）；MBZUAI（穆罕默德·本·扎耶德人工智能大学）；AIRI（人工智能研究院）；FRC CSC RAS（俄罗斯科学院计算机科学与控制研究中心联邦研究中心）；The University of Melbourne（墨尔本大学）；QCRI（卡塔尔计算研究所）；HSE University（高等经济大学）
High dimensional data often contain multiple facets, and several clustering patterns can co-exist under different variable subspaces, also known as the views. While multi-view clustering algorithms were proposed, the uncertainty quantification remains difficult --- a particular challenge is in the high complexity of estimating the cluster assignment probability under each view, and sharing information among views. In this article, we propose an approximate Bayes approach --- treating the similarity matrices generated over the views as rough first-stage estimates for the co-assignment probabili…
Shifting Attention to Relevance: Towards the Predictive Uncertainty Quantification of Free-Form Large Language Models
Drexel University（德雷塞尔大学）；AWS AI Lab（亚马逊云科技人工智能实验室）；Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Lawrence Livermore National Laboratory（劳伦斯利弗莫尔国家实验室）
Probability is Not Enough: Exploring and Counting Divergent Tokens for Reasoning Uncertainty Quantification in LLMs
College of Computer Science and Software Engineering, Shenzhen University（深圳大学计算机与软件学院）；Tsinghua University（清华大学）；Behavioral and Spatial AI Lab, Peking University & Tongji University（北京大学与同济大学行为与空间人工智能实验室）
Texas A&M University（得克萨斯农工大学）；University of Washington（华盛顿大学）
Activity recognition in wearable computing faces two key challenges: i) activity characteristics may be context-dependent and change under different contexts or situations; ii) unknown contexts and activities may occur from time to time, requiring flexibility and adaptability of the algorithm. We develop a context-aware mixture of deep models termed the α-\b{eta} network coupled with uncertainty quantification (UQ) based upon maximum entropy to enhance human activity recognition performance. We improve accuracy and F score by 10% by identifying high-level contexts in a data-driven way to guide…
Efficient Latent Semantic Clustering for Scaling Test-Time Computation of LLMs
Department of Computer Science and Engineering, POSTECH, South Korea（计算机科学与工程系，POSTECH，韩国）；Graduate School of Artificial Intelligence, POSTECH, South Korea（人工智能研究生院，POSTECH，韩国）
Department of School of Engineering, Shenzhen MSU-BIT University, Shenzhen, China, 518000（深圳MSU-BIT大学工程学院部门）；MSU-BIT-SMBU Joint Research Center of Applied Mathematics, Shenzhen MSU-BIT University, Shenzhen, China, 518000（应用数学联合研究中心）
Model Accuracy and Data Heterogeneity Shape Uncertainty Quantification in Machine Learning Interatomic Potentials
Department of Materials Science and Engineering, Faculty of Mechanical Engineering, Delft University of Technology（材料科学与工程系，机械工程学院，代尔夫特理工大学）；J. Mike Walker’66 Department of Mechanical Engineering, Texas A&M University（J. Mike Walker’66机械工程系，德克萨斯农工大学）；Department of Materials Science & Engineering, Texas A&M University（材料科学与工程系，德克萨斯农工大学）
Analysis of short text, such as social media posts, is extremely difficult because of their inherent brevity. In addition to classifying topics of such posts, a common downstream task is grouping the authors of these documents for subsequent analyses. We propose a novel model that expands on the Latent Dirichlet Allocation by modeling strong dependence among the words in the same document, with user-level topic distributions. We also simultaneously cluster users, removing the need for post-hoc cluster estimation and improving topic estimation by shrinking noisy user-level topic distributions t…
Glitch Tokens in Large Language Models: Categorization Taxonomy and Effective Detection
Huazhong University of Science and Technology（华中科技大学）；Nanyang Technological University（南洋理工大学）；Virginia Tech（弗吉尼亚理工大学）；The University of New South Wales（新南威尔士大学）
