穿越灰色地带:不确定性表达与过度自信如何影响语言模型
Navigating the Grey Area: How Expressions of Uncertainty and Overconfidence Affect Language Models
AI总结:
本研究探讨语言模型对输入中确定性、不确定性等认识标记的敏感性,发现高确定性表达反而降低准确率,且模型行为可能源于模仿而非真实认识状态。
摘要:
语言模型在涉及知识和事实的现实任务中的部署日益增多,这使得理解模型的认识论变得重要:语言模型认为自己知道什么,以及它们对这些知识的态度如何受到输入中语言使用的影响。在此,我们研究了模型认识论的一个方面:像“我确定它是”、“我认为它是”或“维基百科说它是”这样的确定性、不确定性或证据性的认识标记如何影响模型,以及它们是否导致模型失败。我们开发了一套认识标记的类型学,并将50个标记注入到问答提示中。我们发现,语言模型对提示中的认识标记高度敏感,准确率变化超过80%。令人惊讶的是,我们发现高确定性表达相比低确定性表达导致准确率下降7%;同样,事实性动词损害性能,而证据性表达则有益于性能。我们对一个流行的预训练数据集的分析表明,这些不确定性标记与问答网站上的答案相关联,而确定性标记则与问题相关联。这些关联可能表明语言模型的行为是基于模仿观察到的语言使用,而非真正反映认识上的不确定性。
英文摘要:
The increased deployment of LMs for real-world tasks involving knowledge and facts makes it important to understand model epistemology: what LMs think they know, and how their attitudes toward that knowledge are affected by language use in their inputs. Here, we study an aspect of model epistemology: how epistemic markers of certainty, uncertainty, or evidentiality like "I'm sure it's", "I think it's", or "Wikipedia says it's" affect models, and whether they contribute to model failures. We develop a typology of epistemic markers and inject 50 markers into prompts for question answering. We find that LMs are highly sensitive to epistemic markers in prompts, with accuracies varying more than 80%. Surprisingly, we find that expressions of high certainty result in a 7% decrease in accuracy as compared to low certainty expressions; similarly, factive verbs hurt performance, while evidentials benefit performance. Our analysis of a popular pretraining dataset shows that these markers of uncertainty are associated with answers on question-answering websites, while markers of certainty are associated with questions. These associations may suggest that the behavior of LMs is based on mimicking observed language use, rather than truly reflecting epistemic uncertainty.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。