用语言模型区分可知与不可知
Distinguishing the Knowable from the Unknowable with Language Models
AI总结：
本文研究用语言模型区分认知不确定性与偶然不确定性，通过线性探针和无监督方法证明LLMs内部天然包含不同类型不确定性的表示，可用于改进模型置信度指标。
摘要：
我们研究了在大型语言模型（LLMs）的自由形式文本输出中识别认知不确定性（反映知识缺乏）而非偶然不确定性（反映底层分布中的熵）的可行性。在缺乏真实概率的情况下，我们探索了一种设置：为了（近似地）解开给定LLM的不确定性，使用一个显著更大的模型作为真实值的代理。我们表明，在冻结的预训练模型嵌入上训练的小型线性探针能够准确预测更大的模型在token级别上何时会更有信心，并且在一个文本领域上训练的探针可以泛化到其他领域。进一步地，我们提出了一种完全无监督的方法，在同一任务上取得了非平凡的准确率。综合来看，我们将这些结果解释为证据，表明LLMs天然包含不同类型不确定性的内部表示，这些表示有可能被利用来在各种实际场景中设计更具信息量的模型置信度指标。
英文摘要：
We study the feasibility of identifying epistemic uncertainty (reflecting a lack of knowledge), as opposed to aleatoric uncertainty (reflecting entropy in the underlying distribution), in the outputs of large language models (LLMs) over free-form text. In the absence of ground-truth probabilities, we explore a setting where, in order to (approximately) disentangle a given LLM's uncertainty, a significantly larger model stands in as a proxy for the ground truth. We show that small linear probes trained on the embeddings of frozen, pretrained models accurately predict when larger models will be more confident at the token level and that probes trained on one text domain generalize to others. Going further, we propose a fully unsupervised method that achieves non-trivial accuracy on the same task. Taken together, we interpret these results as evidence that LLMs naturally contain internal representations of different types of uncertainty that could potentially be leveraged to devise more informative indicators of model confidence in diverse practical settings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
