依赖不可靠者:语言模型不愿表达不确定性的影响
Relying on the Unreliable: The Impact of Language Models' Reluctance to Express Uncertainty
AI总结:
本研究揭示公开部署的语言模型不愿表达不确定性、显式提示后仍过度自信(置信回复平均错误率47%)的问题,通过人体实验证实用户高度依赖LM生成内容,发现偏好标注数据集存在不确定性偏见,提出人机交互安全的设计建议与缓解策略。
摘要:
随着自然语言成为人机交互的默认接口,语言模型(LM)需要在下游应用中恰当地传达不确定性。在本研究中,我们探究了LM如何通过自然语言在回复中融入置信度,以及下游用户对LM表述的不确定性会作何反应。我们对公开部署的模型展开研究,发现LM在回答问题时即便给出错误答案,也不愿表达不确定性。可以通过显式提示让LM表达置信度,但它们往往过于自信,导致置信回复中的错误率很高(平均为47%)。我们通过人体实验测试了LM过度自信的风险,结果表明,无论LM生成的内容是否标注了确定性,用户都会高度依赖这些内容。最后,我们研究了用于后训练对齐的偏好标注数据集,发现人类对带有不确定性的文本存在偏见。本研究揭示了人机交互面临的新型安全危害,并提出了后续的设计建议与缓解策略。
英文摘要:
As natural language becomes the default interface for human-AI interaction, there is a need for LMs to appropriately communicate uncertainties in downstream applications. In this work, we investigate how LMs incorporate confidence in responses via natural language and how downstream users behave in response to LM-articulated uncertainties. We examine publicly deployed models and find that LMs are reluctant to express uncertainties when answering questions even when they produce incorrect responses. LMs can be explicitly prompted to express confidences, but tend to be overconfident, resulting in high error rates (an average of 47%) among confident responses. We test the risks of LM overconfidence by conducting human experiments and show that users rely heavily on LM generations, whether or not they are marked by certainty. Lastly, we investigate the preference-annotated datasets used in post training alignment and find that humans are biased against texts with uncertainty. Our work highlights new safety harms facing human-LM interactions and proposes design recommendations and mitigating strategies moving forward.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。