Uncertainty Quantification in LLM Agents: Foundations, Emerging Challenges, and Opportunities
University of Wisconsin–Madison（威斯康星大学麦迪逊分校）；Carnegie Mellon University（卡内基梅隆大学）；Nanyang Technological University（南洋理工大学）；University of Pennsylvania（宾夕法尼亚大学）；University of Southern California（南加州大学）；University of California, Berkeley（加州大学伯克利分校）
Changdae Oh, Seongheon Park, To Eun Kim, Jiatong Li, Wendi Li, Samuel Yeh, Xuefeng Du, Hamed Hassani, Paul Bogdan, Dawn Song, Sharon Li
Uncertainty quantification (UQ) for large language models (LLMs) is a key building block for safety guardrails of daily LLM applications. Yet, even as LLM agents are increasingly deployed in highly complex tasks, most UQ research still centers on single-turn question-answering. We argue that UQ research must shift to realistic settings with interactive agents, and that a new principled framework for agent UQ is needed. This paper presents three pillars to build a solid ground for future agent UQ research: (1. Foundations) We present the first general formulation of agent UQ that subsumes broad classes of existing UQ setups; (2. Challenges) We identify four technical challenges specifically tied to agentic setups -- selection of uncertainty estimator, uncertainty of heterogeneous entities, modeling uncertainty dynamics in interactive systems, and lack of fine-grained benchmarks -- with numerical analysis on a real-world agent benchmark, $τ^2$-bench; (3. Future Directions) We conclude with noting on the practical implications of agent UQ and remaining open problems as forward-looking discussion for future explorations.
Emerging Opportunities of Using Large Language Models for Translation Between Drug Molecules and Indications
University of Pittsburgh（匹兹堡大学）；Weill Cornell Medicine（威尔康奈尔医学院）；Jerzy Haber Institute of Catalysis and Surface Chemistry, Polish Academy of Sciences（波兰科学院耶日·哈伯催化与表面化学研究所）
DiTTO-LLM: Framework for Discovering Topic-based Technology Opportunities via Large Language Model
Department of Library and Information Science（图书馆与信息科学系）；Chung-Ang University（Chung-Ang 大学）；Insurance Strategy Planning Team（保险战略规划团队）；Postal Savings & Insurance Development Institute（邮政储蓄与保险发展研究所）；Future Technology Analysis Center（未来技术分析中心）；Korea Institute of Science and Technology Information（韩国科学技术信息院）
From Passive Metric to Active Signal: The Evolving Role of Uncertainty Quantification in Large Language Models
Salesforce AI Research（Salesforce AI研究院）；Intuit（Intuit公司）；Vanderbilt University（范德比大学）；University of California, Davis（加州大学戴维斯分校）；Vanderbilt University Medical Center（范德比大学医学中心）
University of Rochester（罗切斯特大学）；Massachusetts Institute of Technology（麻省理工学院）；Harvard University（哈佛大学）
Artificial intelligence (AI) literacy is a rapidly growing research area and a critical addition to K-12 education. However, support for designing tools and curriculum to teach K-12 AI literacy is still limited. There is a need for additional interdisciplinary human-computer interaction and education research investigating (1) how general AI literacy is currently implemented in learning experiences and (2) what additional guidelines are required to teach AI literacy in specifically K-12 learning contexts. In this paper, we analyze a collection of K-12 AI and education literature to show how co…
Artificial Intelligence (AI) has become an integral part of domains such as security, finance, healthcare, medicine, and criminal justice. Explaining the decisions of AI systems in human terms is a key challenge--due to the high complexity of the model, as well as the potential implications on human interests, rights, and lives . While Explainable AI is an emerging field of research, there is no consensus on the definition, quantification, and formalization of explainability. In fact, the quantification of explainability is an open challenge. In our previous work, we incorporated domain knowle…
National Institute of Technology Silchar（锡尔查尔国家理工学院）；Ericsson（爱立信）
Edge technology aims to bring Cloud resources (specifically, the compute, storage, and network) to the closed proximity of the Edge devices, i.e., smart devices where the data are produced and consumed. Embedding computing and application in Edge devices lead to emerging of two new concepts in Edge technology, namely, Edge computing and Edge analytics. Edge analytics uses some techniques or algorithms to analyze the data generated by the Edge devices. With the emerging of Edge analytics, the Edge devices have become a complete set. Currently, Edge analytics is unable to provide full support fo…
Embodied AI: Emerging Risks and Opportunities for Policy Action
Centre for the Governance of AI（治理人工智能中心）；French Center for AI Safety (CeSIA)（法国人工智能安全中心）；Carnegie Mellon University（卡内基梅隆大学）；University of Oxford（牛津大学）；University of Bologna（博洛尼亚大学）；Tony Blair Institute for Global Change（托尼·布莱尔全球变革研究所）；Yale University（耶鲁大学）；Yale Digital Ethics Center（耶鲁数字伦理中心）
Developing knowledge-driven contemporaneous health index (CHI) that can precisely reflect the underlying patient across the course of the condition's progression holds a unique value, like facilitating a range of clinical decision-making opportunities. This is particularly important for monitoring degenerative condition such as Alzheimer's disease (AD), where the condition of the patient will decay over time. Detecting early symptoms and progression sign, and continuous severity evaluation, are all essential for disease management. While a few methods have been developed in the literature, unc…
Single- to multi-fidelity history-dependent learning with uncertainty quantification and disentanglement: application to data-driven constitutive modeling
Faculty of Mechanical Engineering Delft University of Technology（代尔夫特理工大学机械工程学院）；School of Engineering Brown University（布朗大学工程学院）
Lawrence Livermore National Laboratory（劳伦斯利弗莫尔国家实验室）
The emerging paradigm of Human-Machine Inference Networks (HuMaINs) combines complementary cognitive strengths of humans and machines in an intelligent manner to tackle various inference tasks and achieves higher performance than either humans or machines by themselves. While inference performance optimization techniques for human-only or sensor-only networks are quite mature, HuMaINs require novel signal processing and machine learning solutions. In this paper, we present an overview of the HuMaINs architecture with a focus on three main issues that include architecture design, inference algo…
A Comparative Evaluation of Quantification Methods
University of Mannheim（曼海姆大学）；RWTH Aachen University（亚琛工业大学）；GESIS - Leibniz Institute for the Social Sciences（GESIS莱布尼茨社会科学研究所）；Complexity Science Hub（复杂性科学中心）；University of Passau（帕绍大学）
EPFL（洛桑联邦理工学院）；Sharif University of Technology（谢里夫理工大学）
Neuroscience and neurotechnology are currently being revolutionized by artificial intelligence (AI) and machine learning. AI is widely used to study and interpret neural signals (analytical applications), assist people with disabilities (prosthetic applications), and treat underlying neurological symptoms (therapeutic applications). In this brief, we will review the emerging opportunities of on-chip AI for the next-generation implantable brain-machine interfaces (BMIs), with a focus on state-of-the-art prosthetic BMIs. Major technological challenges for the effectiveness of AI models will be d…
A Critical Synthesis of Uncertainty Quantification and Foundation Models in Monocular Depth Estimation
Institute of Photogrammetry and Remote Sensing (IPF), Karlsruhe Institute of Technology (KIT)（摄影测量与遥感研究所（卡尔斯鲁厄理工学院））；The Ohio State University（俄亥俄州立大学）
Federated learning (FL) has been developed as a promising framework to leverage the resources of edge devices, enhance customers' privacy, comply with regulations, and reduce development costs. Although many methods and applications have been developed for FL, several critical challenges for practical FL systems remain unaddressed. This paper provides an outlook on FL development, categorized into five emerging directions of FL, namely algorithm foundation, personalization, hardware and security constraints, lifelong learning, and nonstandard data. Our unique perspectives are backed by practic…
Uncertainty quantification for probabilistic machine learning in earth observation using conformal prediction
Stellenbosch University（斯泰伦博斯大学）；The Nature Conservancy（大自然保护协会）；University of Cape Town（开普敦大学）；Norwegian Institute for Nature Research—NINA（挪威自然研究所—NINA）；Jet Propulsion Laboratory（喷气推进实验室）；California Institute of Technology（加州理工学院）；South African Environmental Observation Network（南非环境观测网络）
Internet of Things (IoT) is emerging as a significant technology in shaping the future by connecting physical devices or things with internet. It also presents various opportunities for intersection of other technological trends which can allow it to become even more intelligent and efficient. In this paper we focus our attention on the integration of Intelligent Conversational Software Agents or Chatbots with IoT. Literature surveys have looked into various applications, features, underlying technologies and known challenges of IoT. On the other hand, Chatbots are being adopted in greater num…
Prediction of wind turbines power with physics-informed neural networks and evidential uncertainty quantification
University of Granada（格拉纳达大学）；Research Centre for Information and Communication Technologies (CITIC-UGR)（信息与通信技术研究中心（CITIC-UGR））；TECNALIA（泰克纳利亚研究中心）；Imperial College London（帝国理工学院）
There exist situations of decision-making under information overload in the Internet, where people have an overwhelming number of available options to choose from, e.g. products to buy in an e-commerce site, or restaurants to visit in a large city. Recommender systems arose as a data-driven personalized decision support tool to assist users in these situations: they are able to process user-related data, filtering and recommending items based on the users preferences, needs and/or behaviour. Unlike most conventional recommender approaches where items are inanimate entities recommended to the u…
German Cancer Research Center（德国癌症研究中心）；Heidelberg University（海德堡大学）；University of Bern（伯尔尼大学）
Multispectral photoacoustic imaging (PAI) is an emerging imaging modality which enables the recovery of functional tissue parameters such as blood oxygenation. However, the underlying inverse problems are potentially ill-posed, meaning that radically different tissue properties may - in theory - yield comparable measurements. In this work, we present a new approach for handling this specific type of uncertainty by leveraging the concept of conditional invertible neural networks (cINNs). Specifically, we propose going beyond commonly used point estimates for tissue oxygenation and converting si…
Federated learning (FL) is an emerging technology that enables the training of machine learning models from multiple clients while keeping the data distributed and private. Based on the participating clients and the model training scale, federated learning can be classified into two types: cross-device FL where clients are typically mobile devices and the client number can reach up to a scale of millions; cross-silo FL where clients are organizations or companies and the client number is usually small (e.g., within a hundred). While existing studies mainly focus on cross-device FL, this paper …
Recent advances in Internet of Things (IoT) technologies and the reduction in the cost of sensors have encouraged the development of smart environments, such as smart homes. Smart homes can offer home assistance services to improve the quality of life, autonomy and health of their residents, especially for the elderly and dependent. To provide such services, a smart home must be able to understand the daily activities of its residents. Techniques for recognizing human activity in smart homes are advancing daily. But new challenges are emerging every day. In this paper, we present recent algori…
