Selective prediction systems can mitigate harms resulting from language model hallucinations by abstaining from answering in high-risk cases. Uncertainty quantification techniques are often employed to identify such cases, but are rarely evaluated in the context of the wider selective prediction policy and its ability to operate at low target error rates. We identify a model-dependent failure mode of entropy-based uncertainty methods that leads to unreliable abstention behaviour, and address it by combining entropy scores with a correctness probe signal. We find that across three QA benchmarks (TriviaQA, BioASQ, MedicalQA) and four model families, the combined score generally improves both the risk--coverage trade-off and calibration performance relative to entropy-only baselines. Our results highlight the importance of deployment-facing evaluation of uncertainty methods, using metrics that directly reflect whether a system can be trusted to operate at a stated risk level.
Train No Evil: Selective Masking for Task-Guided Pre-Training
Department of Computer Science and Technology, Tsinghua University（清华大学计算机科学与技术系）；Institute for Artificial Intelligence, Tsinghua University（清华大学人工智能研究院）；State Key Lab on Intelligent Technology and Systems, Tsinghua University（清华大学智能技术与系统国家重点实验室）
A BERTology View of LLM Orchestrations: Token- and Layer-Selective Probes for Efficient Single-Pass Classification
Departamento de Computación, FCEyN Universidad de Buenos Aires（布宜诺斯艾利斯大学计算机系）；ELIAS Lab, Departamento de Ingeniería Universidad de San Andres（圣安德烈斯大学工程系ELIAS实验室）
DriftGuard: Safety-Aware Multi-Monitor Detection and Selective Adaptation for Evolving Toxicity Moderation
Department of Information and Decision Sciences（信息与决策科学系）；University of Minnesota（明尼苏达大学）；Department of Industrial Engineering and Management Sciences（工业工程与管理科学系）；Northwestern University（西北大学）；Fuqua School of Business（福卡商学院）；Duke University（杜克大学）；Department of Engineering（工程系）；Carnegie Mellon University（卡内基梅隆大学）
Inference-Time Selective Debiasing to Enhance Fairness in Text Classification Models
MBZUAI（穆罕默德·本·扎耶德人工智能大学）；AIRI（人工智能研究所）；RUDN University（俄罗斯人民友谊大学）；FRC CSC RAS（俄罗斯科学院联邦研究中心计算机科学与控制研究所）；IIIT Delhi（德里印度信息技术学院）；The University of Melbourne（墨尔本大学）
