随机有多随机？评估 LLM 抛硬币的随机性与人类特征
Heads, Tails, and AI Fails: LLMs, Randomness, and Human Judgments
AI总结：
本研究通过生成二元随机序列的视角，探讨 LLM 如何处理随机性及其失败方式，发现 GPT 4 和 Llama 3 加剧了人类偏见，而 GPT 3.5 更随机，并提出随机性与人类特征的二分法是 LLM 的基本问题。
摘要：
人类的一个独特特征是无法做到随机。我们在不该有模式的地方看到并产生模式，且以可预测的方式这样做。LLM 由人类数据提供并易受人类偏见影响。在本研究中，我们通过研究充分的生成二元随机序列现象这一视角，探讨 LLM 如何处理随机性以及它们在何处及如何失败。我们发现 GPT 4 和 Llama 3 在此情境中表现出并加剧了我们测试的几乎所有人类偏见，但 GPT 3.5 表现出更随机的行为。这种随机性或人类特征的二分法被提出作为 LLM 的一个基本问题，且两种行为在不同情况下可能都有用。
英文摘要：
Randomness is central to human cognition and to many applications in which large language models are deployed, yet probabilistic token generation does not imply that LLMs can produce unbiased random sequences. We study how contemporary LLMs generate binary random sequences using the classic behavioral-science paradigm of simulated coin flips. Across single flips, 20-flip sequences, n-gram statistics, run lengths, alternation rates, and next-flip predictability, we compare model outputs to both true Bernoulli baselines and human data from prior work. We find that LLMs reproduce several canonical human randomness biases, including over-alternation, aversion to long runs, and first-flip biases, but often amplify them or introduce model-specific distortions. Increasing temperature reduces some rigid patterns but does not eliminate systematic structure. We further investigate over-alternation through prompt-framing experiments, continuation tests, corpus searches, and residual-stream probes, finding evidence that this bias is not explained by direct memorization or tokenization alone. These results show that LLMs are unreliable generators of randomness and imperfect simulators of human randomness judgments, with implications for applications requiring stochasticity, fair sampling, or behavioral simulation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
