Uncertainty quantification in Large Language Models (LLMs) is crucial for applications where safety and reliability are important. In particular, uncertainty can be used to improve the trustworthiness of LLMs by detecting factually incorrect model responses, commonly called hallucinations. Critically, one should seek to capture the model's semantic uncertainty, i.e., the uncertainty over the meanings of LLM outputs, rather than uncertainty over lexical or syntactic variations that do not affect answer correctness. To address this problem, we propose Kernel Language Entropy (KLE), a novel method for uncertainty estimation in white- and black-box LLMs. KLE defines positive semidefinite unit trace kernels to encode the semantic similarities of LLM outputs and quantifies uncertainty using the von Neumann entropy. It considers pairwise semantic dependencies between answers (or semantic clusters), providing more fine-grained uncertainty estimates than previous methods based on hard clustering of answers. We theoretically prove that KLE generalizes the previous state-of-the-art method called semantic entropy and empirically demonstrate that it improves uncertainty quantification performance across multiple natural language generation datasets and LLM architectures.
Fine-Grained Uncertainty Quantification via Collisions
Department of Electrical and Computer Engineering, The University of Arizona（电气与计算机工程系，亚利桑那大学）；Pacific Northwest National Laboratory（太平洋西北国家实验室）；Marvell Technology Inc.（Marvell技术公司）
Lawrence Berkeley National Laboratory（劳伦斯伯克利国家实验室）；Los Alamos National Laboratory（洛斯阿拉莫斯国家实验室）
The modeling and uncertainty quantification of closed curves is an important problem in the field of shape analysis, and can have significant ramifications for subsequent statistical tasks. Many of these tasks involve collections of closed curves, which often exhibit structural similarities at multiple levels. Modeling multiple closed curves in a way that efficiently incorporates such between-curve dependence remains a challenging problem. In this work, we propose and investigate a multiple-output (a.k.a. multi-output), multi-dimensional Gaussian process modeling framework. We illustrate the p…
Term clustering is important in biomedical knowledge graph construction. Using similarities between terms embedding is helpful for term clustering. State-of-the-art term embeddings leverage pretrained language models to encode terms, and use synonyms and relation knowledge from knowledge graphs to guide contrastive learning. These embeddings provide close embeddings for terms belonging to the same concept. However, from our probing experiments, these embeddings are not sensitive to minor textual differences which leads to failure for biomedical term clustering. To alleviate this problem, we ad…
Quantification of Large Language Model Distillation
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Peking University（北京大学）；SUSTech（南方科技大学）；SUAT；Leibowitz AI；UNSW Sydney（新南威尔士大学悉尼分校）
Indiana University Bloomington（印第安纳大学伯明顿分校）；Uppsala University（乌普萨拉大学）；Linköping University（林雪平大学）
Diffusion MRI (dMRI) is the only non-invasive technique sensitive to tissue micro-architecture, which can, in turn, be used to reconstruct tissue microstructure and white matter pathways. The accuracy of such tasks is hampered by the low signal-to-noise ratio in dMRI. Today, the noise is characterized mainly by visual inspection of residual maps and estimated standard deviation. However, it is hard to estimate the impact of noise on downstream tasks based only on such qualitative assessments. To address this issue, we introduce a novel metric, Noise Uncertainty Quantification (NUQ), for quanti…
Uncertainty Quantification in LLM Agents: Foundations, Emerging Challenges, and Opportunities
University of Wisconsin–Madison（威斯康星大学麦迪逊分校）；Carnegie Mellon University（卡内基梅隆大学）；Nanyang Technological University（南洋理工大学）；University of Pennsylvania（宾夕法尼亚大学）；University of Southern California（南加州大学）；University of California, Berkeley（加州大学伯克利分校）
ENSTA Paris, Institut polytechnique de Paris（巴黎综合理工学院巴黎高等先进技术学院）；SATIE, Université Paris-Sud, Université Paris-Saclay（巴黎萨克雷大学巴黎南大学SATIE实验室）；CNRS, LIS, Aix Marseille University（艾克斯-马赛大学LIS实验室法国国家科学研究中心）；LTCI, Télécom Paris, Institut polytechnique de Paris（巴黎综合理工学院巴黎电信学院LTCI实验室）
Deep neural networks (DNNs) are powerful learning models yet their results are not always reliable. This is due to the fact that modern DNNs are usually uncalibrated and we cannot characterize their epistemic uncertainty. In this work, we propose a new technique to quantify the epistemic uncertainty of data easily. This method consists in mixing the predictions of an ensemble of DNNs trained to classify One class vs All the other classes (OVA) with predictions from a standard DNN trained to perform All vs All (AVA) classification. On the one hand, the adjustment provided by the AVA DNN to the …
The data-centric construction of inexpensive surrogates for fine-grained, physical models has been at the forefront of computational physics due to its significant utility in many-query tasks such as uncertainty quantification. Recent efforts have taken advantage of the enabling technologies from the field of machine learning (e.g. deep neural networks) in combination with simulation data. While such strategies have shown promise even in higher-dimensional problems, they generally require large amounts of training data even though the construction of surrogates is by definition a Small Data pr…
Detecting Misbehaviors of Large Vision-Language Models by Evidential Uncertainty Quantification
State Key Laboratory of Advanced Rail Autonomous Operation（先进轨道交通自主运行国家重点实验室）；Beijing Key Laboratory of Traffic Data Mining and Embodied Intelligence（北京交通数据挖掘与具身智能重点实验室）；School of Computer Science and Technology, Beijing Jiaotong University（北京交通大学计算机科学与技术学院）；School of Automation and Intelligence, Beijing Jiaotong University（北京交通大学自动化与智能学院）；Beijing Key Laboratory of Security and Privacy in Intelligent Transportation（北京智能交通安全与隐私重点实验室）
The automated construction of coarse-grained models represents a pivotal component in computer simulation of physical systems and is a key enabler in various analysis and design tasks related to uncertainty quantification. Pertinent methods are severely inhibited by the high-dimension of the parametric input and the limited number of training input/output pairs that can be generated when computationally demanding forward models are considered. Such cases are frequently encountered in the modeling of random heterogeneous media where the scale of the microstructure necessitates the use of high-d…
Movie genre classification is an active research area in machine learning. However, due to the limited labels available, there can be large semantic variations between movies within a single genre definition. We expand these 'coarse' genre labels by identifying 'fine-grained' semantic information within the multi-modal content of movies. By leveraging pre-trained 'expert' networks, we learn the influence of different combinations of modes for multi-label genre classification. Using a contrastive loss, we continue to fine-tune this 'coarse' genre classification network to identify high-level in…
FiLo: Zero-Shot Anomaly Detection by Fine-Grained Description and High-Quality Localization
Foundation Model Research Center, Institute of Automation, Chinese Academy of Sciences, Beijing, China（中国科学院自动化研究所基础模型研究中心）；University of Chinese Academy of Sciences, Beijing, China（中国科学院大学）；Objecteye Inc., Beijing, China（Objecteye公司）；Central South University, Hunan, China（中南大学）
Australian National University（澳大利亚国立大学）；Data61-Commonwealth Scientific Industrial Research Organization（Data61-联邦科学与工业研究组织）；Commonwealth Scientific Industrial Research Organization（联邦科学与工业研究组织）；The University of Technology Sydney（悉尼科技大学）
To make the best use of the underlying minute and subtle differences, fine-grained classifiers collect information about inter-class variations. The task is very challenging due to the small differences between the colors, viewpoint, and structure in the same class entities. The classification becomes more difficult due to the similarities between the differences in viewpoint with other classes and differences with its own. In this work, we investigate the performance of the landmark general CNN classifiers, which presented top-notch results on large scale classification datasets, on the fine-…
Massachusetts Institute of Technology（麻省理工学院）；University of Florida（佛罗里达大学）；Northeastern University（东北大学）
Origin-Destination (O-D) travel demand prediction is a fundamental challenge in transportation. Recently, spatial-temporal deep learning models demonstrate the tremendous potential to enhance prediction accuracy. However, few studies tackled the uncertainty and sparsity issues in fine-grained O-D matrices. This presents a serious problem, because a vast number of zeros deviate from the Gaussian assumption underlying the deterministic deep learning models. To address this issue, we design a Spatial-Temporal Zero-Inflated Negative Binomial Graph Neural Network (STZINB-GNN) to quantify the uncert…
In this paper, we present PARTICUL, a novel algorithm for unsupervised learning of part detectors from datasets used in fine-grained recognition. It exploits the macro-similarities of all images in the training set in order to mine for recurring patterns in the feature space of a pre-trained convolutional neural network. We propose new objective functions enforcing the locality and unicity of the detected parts. Additionally, we embed our detectors with a confidence measure based on correlation scores, allowing the system to estimate the visibility of each part. We apply our method on two publ…
Metric-based few-shot fine-grained image classification (FSFGIC) aims to learn a transferable feature embedding network by estimating the similarities between query images and support classes from very few examples. In this work, we propose, for the first time, to introduce the non-linear data projection concept into the design of FSFGIC architecture in order to address the limited sample problem in few-shot learning and at the same time to increase the discriminability of the model for fine-grained image classification. Specifically, we first design a feature re-abstraction embedding network …
