当LLMs玩电话游戏：文化吸引子作为评估多轮设置中LLM的观念工具
When LLMs Play the Telephone Game: Cultural Attractors as Conceptual Tools to Evaluate LLMs in Multi-turn Settings
AI总结：
本文通过电话游戏实验揭示LLM多轮交互中的信息失真与吸引子现象，探讨初始文本、指令等对吸引子效应的影响。
摘要：
随着大型语言模型（LLMs）开始相互交互并生成越来越多的在线文本，理解信息如何在从一个LLM传递到下一个LLM的过程中发生变化变得至关重要。尽管已有大量研究探讨了单个LLM的行为，但现有研究大多忽视了由迭代LLM交互产生的集体行为和信息失真。微小的偏见，在单个输出层面可能显得无关紧要，但在迭代交互中可能会被放大，使内容朝着吸引子状态发展。在一系列电话游戏实验中，我们应用了从人类文化进化文献中借来的传输链设计：LLM代理在链中依次接收、生成和传递文本。通过跟踪文本毒性、积极性、难度和长度在传输链中的演变，我们揭示了偏见和吸引子的存在，并研究其对初始文本、指令、语言模型和模型大小的依赖性。例如，我们发现更开放的指令比更受限的任务导致更强的吸引力效应。我们还发现不同的文本属性对吸引力效应的敏感性不同，毒性比长度产生更强的吸引子。这些发现突显了考虑多步传输动态的重要性，并代表了对LLM文化动态更全面理解的第一步。
英文摘要：
As large language models (LLMs) start interacting with each other and generating an increasing amount of text online, it becomes crucial to better understand how information is transformed as it passes from one LLM to the next. While significant research has examined individual LLM behaviors, existing studies have largely overlooked the collective behaviors and information distortions arising from iterated LLM interactions. Small biases, negligible at the single output level, risk being amplified in iterated interactions, potentially leading the content to evolve towards attractor states. In a series of telephone game experiments, we apply a transmission chain design borrowed from the human cultural evolution literature: LLM agents iteratively receive, produce, and transmit texts from the previous to the next agent in the chain. By tracking the evolution of text toxicity, positivity, difficulty, and length across transmission chains, we uncover the existence of biases and attractors, and study their dependence on the initial text, the instructions, language model, and model size. For instance, we find that more open-ended instructions lead to stronger attraction effects compared to more constrained tasks. We also find that different text properties display different sensitivity to attraction effects, with toxicity leading to stronger attractors than length. These findings highlight the importance of accounting for multi-step transmission dynamics and represent a first step towards a more comprehensive understanding of LLM cultural dynamics.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
