Large language models (LLMs) are empowering decision-making in several applications, including tool or API usage and answering multiple-choice questions (MCQs). However, incorrect outputs pose significant risks in high-stakes domains like healthcare and finance. To quantify LLM uncertainty and thereby mitigate these risks, recent works employ conformal prediction (CP), a model- and distribution-agnostic framework that uses LLM outputs to generate a \emph{prediction set} containing the true answer with high probability. Leveraging CP, we propose \emph{conformal revision of questions} (CROQ), which revises the question by narrowing down the available choices to those in the prediction set and asking the LLM the revised question. We expect LLMs to be more accurate on revised questions with fewer choices. Furthermore, we expect CROQ to be effective when the prediction sets from CP are small. Commonly used logit scores often lead to large sets, diminishing CROQ's effectiveness. To overcome this, we propose CP-OPT, an optimization framework to learn scores that minimize set sizes while maintaining coverage. Our extensive experiments on MMLU, ToolAlpaca, and TruthfulQA datasets with multiple LLMs show that CROQ improves accuracy over the standard inference, with more pronounced gains when paired with CP-OPT.
Multi-Variable Conformal Prediction: Optimizing Prediction Sets using Multiple Calibration Variables
TU Munich & MCML, Germany(慕尼黑技术大学及MCML,德国);Politecnico di Milano, Italy(米兰理工学院,意大利);University of Brescia, Italy(布雷西亚大学,意大利);ETH Zürich, Switzerland(苏黎世联邦理工学院,瑞士)
Pruning By Explaining Revisited: Optimizing Attribution Methods to Prune CNNs and Transformers
Fraunhofer Heinrich-Hertz-Institute(弗劳恩霍夫海因茨·赫兹研究所);Technische Universität Berlin(柏林工业大学);BIFOLD – Berlin Institute for the Foundations of Learning and Data(柏林学习与数据基础研究所(BIFOLD))
RoRA: Efficient Fine-Tuning of LLM with Reliability Optimization for Rank Adaptation
Northeastern University(东北大学);Carnegie Mellon University(卡内基梅隆大学);Massachusetts Institute of Technology(麻省理工学院);University of Georgia(佐治亚大学);Florida International University(佛罗里达国际大学)
Department of Systems Engineering, City University of Hong Kong, Hong Kong SAR, China(系统工程系,香港城市大学,香港特别行政区,中国);Alpha Benito Research, Los Angeles, USA(Alpha Benito研究机构,洛杉矶,美国)
Improving the interpretability of GNN predictions through conformal-based graph sparsification
Max Planck Institute for Intelligent Systems(马克斯·普朗克智能系统研究所);McGill University(麦吉尔大学);Saarland University(萨尔大学);Max Planck Institute for Software Systems(马克斯·普朗克软件系统研究所)
Lost in Sampling: Assessing Lexical Reachability in LLMs via the Word Coverage Score (WCS)
Information and Processing Telecommunications Center(信息与处理电信中心);Universidad Politécnica de Madrid(马德里理工大学);Politecnico di Milano(米兰理工大学);Banco de España(西班牙银行)
Defense Against LLM Backdoors using Critical Neuron Isolation Pruning
Huazhong University of Science and Technology(华中科技大学);Nanjing University of Aeronautics and Astronautics(南京航空航天大学);Nanyang Technological University(南洋理工大学)
Uncertainty-Aware Generation and Decision-Making Under Ambiguity
Ubiquitous Knowledge Processing Lab (UKP Lab), Department of Computer Science, Technical University of Darmstadt(普遍知识处理实验室(UKP实验室),计算机科学系,达姆施塔特技术大学);National Research Center for Applied Cybersecurity ATHENE, Germany(应用网络安全国家研究中心ATHENE,德国)