AI 中文总结
研究政治立场一致和不一致的大语言模型事实核查信息的有效性，通过两个受试者内实验发现，LLM能显著改变对新闻标题的信任度，政治一致性在标题政治距离远时起作用，且LLM出错时也会影响信任度，揭示其纠正与歪曲信息的潜力。
AI 中文摘要
社交媒体公司已不再依赖人工事实核查员，而是在其平台上嵌入了对话式大语言模型（LLM）。LLM聊天机器人在许多方面与人工事实核查员不同，这可能会影响用户对纠正信息的反应。本研究特别关注的是，LLM聊天机器人可以通过其回复中强调的内容、引用的来源和设定的角色在意识形态上进行配置。本文利用两个受试者内实验（n = 705）的数据，研究了意识形态配置的LLM聊天机器人的事实核查信息的有效性。我们发现，即使聊天机器人与用户在政治立场上不一致，LLM事实核查员也能显著改变对真假政治新闻标题的信任度。参与者与机器人之间感知到的政治一致性仅在标题政治距离较远时才起作用。也就是说，当政治距离较远的聊天机器人核查距离较远的标题时，对正确标记为真的标题的信任度增加较少，而当中立的聊天机器人核查距离较远的标题时，信任度增加较多。LLM聊天机器人的政治一致性感知并未影响其降低对虚假标题信任度的有效性。不幸的是，当LLM事实核查员出错或提供不确定答案时，它们也会显著改变对新闻的信任度。我们的结果既证明了大语言模型大规模纠正错误信息的潜力，也证明了它们大规模歪曲事实的潜力。
英文摘要
Social media companies have shifted away from human fact-checkers and instead have embedded conversational Large Language Models (LLM) on their platforms. LLM chatbots differ from human fact-checkers in many ways that may shape user responses to corrections. Of particular interest in this study is that LLM chatbots can be ideologically configured via the content emphasized in their responses, the sources cited, and the configured persona. Using data from two within-subjects experiments (n=705), this paper investigates the effectiveness of fact checking information from ideologically configured LLM chatbots. We find that LLM fact-checkers significantly shift trust in true and false political news headlines, even when the chatbot is politically incongruent with the user. The perceived political congruency between the participant and the bot matters only when headlines are politically distant. That is, trust in correctly labeled true headlines increases less when politically distant chatbots check distant headlines and increases more when moderate chatbots check distant headlines. The perceived political congruency of LLM chatbots did not impact their effectiveness at decreasing trust in false headlines. Unfortunately, LLM fact-checkers also significantly change trust in news when they are wrong or provide inconclusive answers. Our results demonstrate both the potential for LLMs to correct false information at scale but also their potential to taint the truth at scale.
CommentsTo Appear at ICWSM 2027
