“不太可能”到底有多“不可能”？评估大型语言模型的言语概率感知
How Unlikely Is "Unlikely"? Assessing Verbal Probability Perception Across Large Language Models
浏览论文内容
中文总结 AI 辅助
该研究通过跨19个LLM的词-数映射等测试，发现LLM能追踪人类言语概率认知结构但负向表达存在偏差，为人机概率语言交互提供了参考。
中文摘要 AI 辅助
大型语言模型（LLM）越来越多地生成和解释言语概率表达，但这些表达在不同模型间是否具有一致含义（或是否匹配人类对不确定性的感知）仍未知。我们基于已建立的人类基准开展了词-数映射任务，进行了系统的跨模型评估：在强制单数值响应与解释引出两种条件下，向19个模型呈现11种不确定性表达，同时开展新型双向往返测试以评估内部一致性。LLM以惊人的保真度追踪人类基准：词序被保留，3个锚点被恢复，“possible”（可能）在所有测试表达中表现出最高方差与跨模型分歧，与人类中已记录的双峰解读一致。不过，模型对“unlikely”（不太可能）、“improbable”（极不可能）等负向表达存在系统性向上偏差。解释引出会降低模型内方差，同时增加模型间分歧，以牺牲模型间共识为代价稳定单个模型；往返实验显示出明显分层，前沿模型维持连贯的双向表征。因此，LLM重现了人类言语概率认知的结构，包括其偏差，同时在负向端存在系统性分歧——这对人类与模型交换概率语言的任何场景都具有重要意义。
英文摘要
Large language models increasingly produce and interpret verbal probability expressions, yet whether these expressions carry consistent meaning across models (or match human perceptions of uncertainty) remains unknown. We present a systematic cross-model evaluation using a word-to-number mapping task grounded in established human benchmarks. Eleven uncertainty expressions were presented to 19 models under two conditions, forced single-number response and explanation elicitation, alongside a novel bidirectional roundtrip test of internal consistency. LLMs track the human benchmark with surprising fidelity: word ordering is preserved, three anchor points are recovered, and ``possible'' shows the highest variance and cross-model disagreement of any expression tested, consistent with its documented bimodal interpretation in humans. However, models show a systematic upward bias for negative expressions such as ``unlikely'' and ``improbable.'' Explanation elicitation reduces within-model variance while increasing between-model divergence, stabilizing individual models at the cost of inter-model consensus, and the roundtrip experiment reveals clear stratification, with frontier models maintaining coherent bidirectional representations. LLMs thus reproduce the structure of human verbal probability cognition, including its biases, while diverging systematically at the negative end---with implications for any setting where humans and models exchange probabilistic language.
发表机构
- Temple University（天普大学）
- University of Pittsburgh（匹兹堡大学）
机构由 AI 辅助整理，请以论文原文为准。
