arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.00977cs.CLcs.LG
收藏

单轮信任:通过结构信号提升大语言模型的置信度估计

Trust in One Round: Confidence Estimation for Large Language Models via Structural Signals

The University of Sydney(悉尼大学);University of Technology Sydney(技术大学悉尼)

Pengyue Yang, Jiawen Wen, Haolin Jin, Linghan Huang, Huaming Chen, Ling Chen

AI总结:

通过分析模型最终层隐藏状态轨迹中的多尺度结构信号,提出结构置信度方法,实现单次通过的高效、稳健置信度估计,适用于高社会影响和资源受限的LLM应用。

摘要:

大型语言模型(LLMs)正越来越多地应用于错误可能带来高额社会、科学或安全成本的领域。然而,标准的置信度估计器,如token似然、语义相似度和多样本一致性,在分布偏移、领域专业化文本和计算限制下仍显得脆弱。在本工作中,我们提出了结构置信度,一种单次通过、模型无关的框架,该框架通过从模型最终层隐藏状态轨迹中提取多尺度结构信号来增强输出正确性预测。通过结合频谱、局部变化和全局形状描述符,我们的方法捕捉到了被概率和句子嵌入所遗漏的内部稳定性模式。我们在四个异质基准测试中进行了广泛的跨领域评估——FEVER(事实验证)、SciFact(科学断言)、WikiBio-幻觉(传记一致性)和TruthfulQA(以真实性为导向的问答)。我们的结构置信度框架在AUROC和AUPR方面表现出色,相比已建立的基线方法。更重要的是,不同于需要多次随机生成和辅助模型的采样一致性方法,我们的方法使用单次确定性前向通过,为在具有社会影响和资源受限的LLM应用中实现高效、稳健的后置置信度估计提供了实用基础。

英文摘要:

Large language models (LLMs) are increasingly deployed in domains where errors carry high social, scientific, or safety costs. Yet standard confidence estimators, such as token likelihood, semantic similarity and multi-sample consistency, remain brittle under distribution shift, domain-specialised text, and compute limits. In this work, we present Structural Confidence, a single-pass, model-agnostic framework that enhances output correctness prediction based on multi-scale structural signals derived from a model's final-layer hidden-state trajectory. By combining spectral, local-variation, and global shape descriptors, our method captures internal stability patterns that are missed by probabilities and sentence embeddings. We conduct extensive, cross-domain evaluation across four heterogeneous benchmarks-FEVER (fact verification), SciFact (scientific claims), WikiBio-hallucination (biographical consistency), and TruthfulQA (truthfulness-oriented QA). Our Structural Confidence framework demonstrates strong performance compared with established baselines in terms of AUROC and AUPR. More importantly, unlike sampling-based consistency methods which require multiple stochastic generations and an auxiliary model, our approach uses a single deterministic forward pass, offering a practical basis for efficient, robust post-hoc confidence estimation in socially impactful, resource-constrained LLM applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑