arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2604.02637cs.CL
收藏

通过角色扮演训练LLM:探索AI素养对说服力的影响

Train Yourself as an LLM: Exploring Effects of AI Literacy on Persuasion via Role-playing LLM Training

Institute for Clarity in Documentation(文档清晰研究所);Inria Paris-Rocquencourt(法国国家信息与自动化研究所巴黎-罗康库尔中心);Rajiv Gandhi University(拉吉夫·甘地大学);Tsinghua University(清华大学);Palmer Research Laboratories(帕默研究实验室)

Qihui Fan, Min Ge, Chenyan Jia, Weiyan Shi

AI总结:

本文通过角色扮演训练LLMimic,提升用户AI素养,减少AI说服力效果,并增强诚实与社会责任感。

摘要:

随着大型语言模型(LLMs)的说服力增强,人们的意见和决策可能在各种上下文中被大规模影响。以往的缓解措施(如AI检测器和免责声明)大多将人们视为AI生成信息的被动接受者。为了提供更积极的干预措施,我们引入了LLMimic,一种基于角色扮演的交互式、游戏化的AI素养教程,参与者扮演LLM角色并经历训练流水线的三个关键阶段(预训练、微调和强化学习)。我们进行了一项2×3组间实验(N=274),参与者要么(1)观看AI历史视频(对照组)或(2)与LLMimic互动(实验组),然后参与三个现实的AI说服场景之一:(a)慈善捐赠说服、(b)恶意金钱索取、或(c)酒店推荐。我们的结果表明,LLMimic显著提高了参与者的AI素养(p < .001),减少了各场景中的说服成功(p < .05),并在酒店场景中增强了诚实和社会责任感水平(p < 0.01)。这些发现表明,LLMimic提供了一种可扩展、以人类为中心的方法,以提高AI素养并支持更明智的与说服性AI互动。

英文摘要:

As large language models (LLMs) become increasingly persuasive, there is concern that people's opinions and decisions may be influenced across various contexts at scale. Prior mitigation (e.g., AI detectors and disclaimers) largely treats people as passive recipients of AI-generated information. To provide a more proactive intervention against persuasive AI, we introduce $\textbf{LLMimic}$, a role-play-based, interactive, gamified AI literacy tutorial, where participants assume the role of an LLM and progress through three key stages of the training pipeline (pretraining, SFT, and RLHF). We conducted a $2 \times 3$ between-subjects study ($N = 274$) where participants either (1) watched an AI history video (control) or (2) interacted with LLMimic (treatment), and then engaged in one of three realistic AI persuasion scenarios: (a) charity donation persuasion, (b) malicious money solicitation, or (c) hotel recommendation. Our results show that LLMimic significantly improved participants' AI literacy ($p < .001$), reduced persuasion success across scenarios ($p < .05$), and enhanced truthfulness and social responsibility levels ($p<0.01$) in the hotel scenario. These findings suggest that LLMimic offers a scalable, human-centered approach to improving AI literacy and supporting more informed interactions with persuasive AI.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑