位置：在LLM中的不确定性量化仅仅是无监督聚类
Position: Uncertainty Quantification in LLMs is Just Unsupervised Clustering
AI总结：
本文指出，当前LLM的不确定性量化方法本质上是无监督聚类算法，无法有效评估模型的外部正确性，导致无法检测出自信但错误的回答。文章提出了改进的不确定性量化方法，以确保模型的自信度能可靠地反映现实。
摘要：
不确定性量化（UQ）被广泛认为是部署大型语言模型（LLM）于高风险领域的主要保障。然而，我们主张该领域存在类别错误：主流LLM的UQ方法本质上是无监督聚类算法。我们证明大多数当前方法本质上量化的是模型生成的内部一致性，而不是其外部正确性。因此，当前方法从根本上无法识别事实现实，并无法检测出“自信幻觉”，即模型在稳定但错误的答案上表现出高自信。因此，当前UQ方法在部署模型时可能会产生误导的安全感。具体而言，我们识别出由于对内部状态的依赖而产生的三种关键病理：超参数敏感危机，使部署不安全；内部评估循环，将稳定性与事实混淆；以及缺乏事实基础，迫使依赖不稳定代理指标来评估不确定性。为解决这一困境，我们倡导向UQ方法转变，并为研究界制定研究路线图，以采用更好的评估指标和设置，实施原生不确定性机制的变化，并将验证锚定在客观事实上，确保模型自信度能可靠地反映现实。
英文摘要：
Uncertainty Quantification (UQ) is widely regarded as the primary safeguard for deploying Large Language Models (LLMs) in high-stakes domains. However, we argue that the field suffers from a category error: mainstream UQ methods for LLMs are just unsupervised clustering algorithms. We demonstrate that most current approaches inherently quantify the internal consistency of the model's generations rather than their external correctness. Consequently, current methods are fundamentally blind to factual reality and fail to detect ``confident hallucinations,'' where models exhibit high confidence in stable but incorrect answers. Therefore, the current UQ methods may create a deceptive sense of safety when deploying the models with uncertainty. In detail, we identify three critical pathologies resulting from this dependence on internal state: a hyperparameter sensitivity crisis that renders deployment unsafe, an internal evaluation cycle that conflates stability with truth, and a fundamental lack of ground truth that forces reliance on unstable proxy metrics to evaluate uncertainty. To resolve this impasse, we advocate for a paradigm shift to UQ and outline a roadmap for the research community to adopt better evaluation metrics and settings, implement mechanism changes for native uncertainty, and anchor verification in objective truth, ensuring that model confidence serves as a reliable proxy for reality.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
