发表机构
Stanford Computer Science; Stanford Statistics（斯坦福大学计算机科学系; 斯坦福大学统计学系）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究重新评估语言模型开放式同质性的证据，指出其核心示例不具代表性、零假设不严格且干预结论缺乏支持，并证明提示干预可提升多样性，但未证实或否定人工蜂群思维的存在。
AI 中文摘要
近期研究认为，语言模型在开放式生成中表现出显著的同质性，并将这种行为称为“人工蜂群思维”，认为其对人类创造力构成长期威胁。我们检验了该研究的三个核心结果。首先，其标志性示例是模型对“写一个涉及时间的隐喻”的回答坍缩为两个聚类。可视化、谱分析、聚类和语言模型标签均与这一描述相矛盾。标签记录了每个回答的载体，即其将时间比作什么。我们的回答和原作者的回答均显示一个主导载体，加上大量不同的次要载体尾部。“时间”是我们多样性最低的主题之一，因此该示例是一个有利案例，而非代表性案例。其次，该论文将同质性与一个要求不高的零假设进行比较：对无关提示的回答。在更严格的零假设下（同一提示下表达真正不同想法的回答），20%-32%的此类配对已超过论文0.8的收敛阈值。一个残余效应在该零假设下依然存在。论文的同一提示配对超过0.8的频率大约是我们不同想法配对的两到三倍。该论文所称的同质性大部分是回答同一提示的共享几何结构。其余测量缺乏任何零假设：未收集人类基线，且模型不可区分性统计量没有零假设。第三，该论文得出结论，推理时干预不足以对抗“人工蜂群思维”，并写道“需要在模型训练层面采用更通用的解决方案”。我们表明，这一结论在三个方面缺乏支持，且一种推理时干预（提示）可靠地提高了测量的响应多样性。我们不解决“人工蜂群思维”是否真实存在的问题。我们表明，已发表的证据并未确立它。
英文摘要
Recent research argues that language models exhibit pronounced homogeneity in open-ended generation, framing such behavior as an Artificial Hivemind that poses a long-term threat to human creativity. We examine three of its central results. First, the flagship example is that model responses to "Write a metaphor involving time" collapse into two clusters. Visualization, spectral analysis, clustering, and language model labels all contradict this description. The labels record each response's vehicle, what it compares time to. Our responses and the original authors' own show one dominant vehicle plus a heavy tail of distinct minority vehicles. "Time" is one of our least diverse topics, so the example is a favorable case, not a representative one. Second, the paper measures homogeneity against an undemanding null: responses to unrelated prompts. Under a more demanding null (same-prompt responses expressing genuinely different ideas), 20%-32% of such pairs already exceed the paper's 0.8 convergence threshold. A residual effect survives this null. The paper's same-prompt pairs exceed 0.8 roughly two to three times as often as our different-idea pairs. Much of what the paper calls homogeneity is the shared geometry of answering the same prompt. The remaining measurements lack any null: no human baseline is collected, and the model-indistinguishability statistic has no null. Third, the paper concludes that inference-time interventions are inadequate for combating the Artificial Hivemind, writing that "more generalizable solutions are needed at the model training level." We show that this conclusion is unsupported in three ways, and that an inference-time intervention (prompting) reliably raises measured response diversity. We do not resolve whether the Artificial Hivemind is real. We show that the published evidence does not establish it.
Comments43 pages
