arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2409.06653cs.HC
收藏

社交媒体环境中人类对LLM生成文本内容的感知

Human Perception of LLM-generated Text Content in Social Media Environments

Kristina Radivojevic, Matthew Chou, Karla Badillo-Urquiola, Paul Brenner

AI总结:

研究通过让1000余名参与者区分社交媒体中的人类与LLM帖子,发现人类识别能力较差但存在可辨识模式,并证实文本对话中的恐怖谷效应。

摘要:

新兴技术,尤其是人工智能(AI),更具体地说是大语言模型(LLM),为恶意行为者操纵数字话语提供了强大工具。由于机器人能够生成大量可信文本,LLM有可能影响传统形式的民主参与,例如选民选择、政府调查,甚至是与监管机构的在线沟通。为研究人类对LLM生成内容的感知,我们招募了1000多名参与者,让他们在社交媒体讨论串中尝试区分机器人帖子和人类帖子。我们发现,人类在识别社交媒体用户帖子真实性质方面表现不佳。我们还发现了人类在社交媒体话语中识别LLM生成文本内容的模式。最后,我们在用户感知和识别两方面都观察到文本对话中的恐怖谷效应。这表明,尽管人类在识别过程中表现不佳,但他们在阅读LLM生成内容时仍能感受到不适。

英文摘要:

Emerging technologies, particularly artificial intelligence (AI), and more specifically Large Language Models (LLMs) have provided malicious actors with powerful tools for manipulating digital discourse. LLMs have the potential to affect traditional forms of democratic engagements, such as voter choice, government surveys, or even online communication with regulators; since bots are capable of producing large quantities of credible text. To investigate the human perception of LLM-generated content, we recruited over 1,000 participants who then tried to differentiate bot from human posts in social media discussion threads. We found that humans perform poorly at identifying the true nature of user posts on social media. We also found patterns in how humans identify LLM-generated text content in social media discourse. Finally, we observed the Uncanny Valley effect in text dialogue in both user perception and identification. This indicates that despite humans being poor at the identification process, they can still sense discomfort when reading LLM-generated content.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑