This paper introduces AnomaLLMy, a novel technique for the automatic detection of anomalous tokens in black-box Large Language Models (LLMs) with API-only access. Utilizing low-confidence single-token predictions as a cost-effective indicator, AnomaLLMy identifies irregularities in model behavior, addressing the issue of anomalous tokens degrading the quality and reliability of models. Validated on the cl100k_base dataset, the token set of GPT-4, AnomaLLMy detected 413 major and 65 minor anomalies, demonstrating the method's efficiency with just \$24.39 spent in API credits. The insights from this research are expected to be beneficial for enhancing the robustness of and accuracy of LLMs, particularly in the development and assessment of tokenizers.
Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs
National University of Singapore(新加坡国立大学);École Polytechnique Fédérale de Lausanne(洛桑联邦理工学院);The Hong Kong University of Science and Technology(香港科技大学)
Double-Calibration: Towards Reliable LLMs via Calibrating Knowledge and Reasoning Confidence
School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, China(中山大学计算机科学与工程学院,广州,中国);Department of Computing, The Hong Kong Polytechnic University, Hong Kong SAR(香港理工大学计算机系,香港特别行政区);School of Science and Technology, Hong Kong Metropolitan University, Hong Kong SAR(香港 Metropolitan 大学科技学院,香港特别行政区)
Generating with Confidence: Uncertainty Quantification for Black-box Large Language Models
University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Carle’s Illinois College of Medicine, University of Illinois at Urbana-Champaign(卡尔伊利诺伊医学院,伊利诺伊大学厄巴纳-香槟分校)
Calibrating Large Language Models Using Their Generations Only
Parameter Lab(参数实验室);IT University of Copenhagen(哥本哈根信息技术大学);Pioneer Centre for Artificial Intelligence(人工智能先锋中心);NAVER AI Lab(NAVER人工智能实验室);University of Tübingen(蒂宾根大学);Tübingen AI Center(蒂宾根人工智能中心)
Confidence Elicitation: A New Attack Vector for Large Language Models
National University of Singapore(新加坡国立大学);Institute of Data Science, National University of Singapore(新加坡国立大学数据科学研究所);Institute for Infocomm Research, A*STAR(新加坡科技研究局信息通信研究所);Centre for Frontier AI Research, A*STAR(新加坡科技研究局前沿人工智能研究中心)
Probability is Not Enough: Exploring and Counting Divergent Tokens for Reasoning Uncertainty Quantification in LLMs
College of Computer Science and Software Engineering, Shenzhen University(深圳大学计算机与软件学院);Tsinghua University(清华大学);Behavioral and Spatial AI Lab, Peking University & Tongji University(北京大学与同济大学行为与空间人工智能实验室)
Black-box Model Ensembling for Textual and Visual Question Answering via Information Fusion
Faculty of Computer Science, University of Vienna(维也纳大学计算机科学学院);UniVie Doctoral School Computer Science, University of Vienna(维也纳大学UniVie计算机科学博士学院);Department of Computer Science, Aarhus University(奥胡斯大学计算机科学系);Faculty of Philological and Cultural Studies, University of Vienna(维也纳大学语文与文化研究学院)
School of Computer Science, University of Sydney(悉尼大学计算机科学学院);City University of Hong Kong(香港城市大学);Shanghai Jiao Tong University(上海交通大学);University of Oxford, Department of Engineering Science(牛津大学工程科学系);Department of Engineering Science, University of Oxford(牛津大学工程科学系)
Claim-Level Confidence Calibration for Reliable Decision Making with Large Language Models
Tsinghua University(清华大学);Vulcan Research(伏尔坎研究院);AIFT;Keio Global Research Institute (KGRI)(庆应全球研究所(KGRI));China Mobile Research Institute(中国移动研究院);Zhongguancun Laboratory(中关村实验室)