arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.12804cs.HCcs.AIcs.LG
收藏

通过摩擦力调节语言模型体验

Modulating Language Model Experiences through Frictions

University of Cambridge(剑桥大学);Carnegie Mellon University(卡内基梅隆大学);Princeton University(普林斯顿大学);University of Oxford(牛津大学);Imperial College(帝国理工学院);The Alan Turing Institute(阿兰·图灵研究所);New York University(纽约大学)

Katherine M. Collins, Valerie Chen, Ilia Sucholutsky, Hannah Rose Kirk, Malak Sadek, Holli Sargeant, Ameet Talwalkar, Adrian Weller, Umang Bhatt

AI总结:

针对过度依赖语言模型的风险,本文受行为科学启发提出“选择性摩擦力”机制,通过在用户体验中添加微小障碍(如提示按钮),有效降低了用户点击率和过度依赖且对准确性影响极小,为人机交互设计提供了新思路。

摘要:

语言模型正在改变用户与世界互动的方式。尽管能力惊人,但过度消费语言模型的输出在短期内可能传播未经验证的错误,长期来看可能损害人类批判性思考的能力。我们如何围绕语言模型开发支架以规范更合适的使用?受行为科学干预启发,我们提出为语言模型体验引入选择性摩擦力,以抑制误用。摩擦力涉及对用户体验的微小修改,例如添加一个按钮来阻碍模型访问,并提醒用户相对于模型的自身专长。通过一项真实人类用户研究,我们在多主题问答任务(代表人们可能使用LLM的场景,如教育和信息检索)中观察到,对LLM施加摩擦力后用户行为发生了转变。我们发现,摩擦力通过降低用户的点击率来调节过度依赖,同时对那些主题的准确性影响极小。然而,摩擦力可能产生非预期效果。我们发现在未设置摩擦力的主题上,用户的点击行为也出现了显著差异。我们的贡献推动了进一步的人机行为交互研究,以指导更有效、更恰当的LLM使用。

英文摘要:

Language models are transforming the ways that their users engage with the world. Despite impressive capabilities, over-consumption of language model outputs risks propagating unchecked errors in the short-term and damaging human capabilities for critical thinking in the long-term. How can we develop scaffolding around language models to curate more appropriate use? We propose selective frictions for language model experiences, inspired by behavioral science interventions, to dampen misuse. Frictions involve small modifications to a user's experience, e.g., the addition of a button impeding model access and reminding a user of their expertise relative to the model. Through a user study with real humans, we observe shifts in user behavior from the imposition of a friction over LLMs in the context of a multi-topic question-answering task as a representative task that people may use LLMs for, e.g., in education and information retrieval. We find that frictions modulate over-reliance by driving down users' click rates while minimally affecting accuracy for those topics. Yet, frictions may have unintended effects. We find marked differences in users' click behaviors even on topics where frictions were not provisioned. Our contributions motivate further study of human-AI behavioral interaction to inform more effective and appropriate LLM use.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑