寡头几乎无法在多模型生态系统中引导模型崩溃
A Dominant Supplier Slows Recursive Drift More Than It Steers It
浏览论文内容
中文总结 AI 辅助
本研究通过受控多模型生态系统实验发现，市场集中化（寡头）几乎不影响模型崩溃的速度和终点，崩溃速度主要由供料模型的易感性及其份额加权决定。
中文摘要 AI 辅助
AI生成的文本正回流到下一代模型的训练语料库中。对其进行的递归训练会导致模型崩溃，近期的研究将这一情境扩展到多个模型相互供料的场景——但几乎总是以市场均匀分割为前提，而真实的生成式AI是寡头垄断的。集中化引发了两个担忧：更少、更统一的来源可能使崩溃加速，且后续模型可能被拖向寡头的输出。我们在受控生态系统中对这两点进行了测试：13个开源的1-4B模型形成由3至13个参与者组成的自然生态系统，并注入一个将最高份额推至90%的探针；每一代，每个模型的输出按市场份额混合到一个共享池中，每个模型从干净的基座权重在该池上重新训练，共进行五代。然而，在我们测试的范围内，这两种担忧均未出现；取而代之的是一种不变性。使分割更加不平等几乎不改变崩溃的速度。目的地移动得更少：份额和身份旋钮将五代终点仅移动了所有臂共同漂移的几个百分点——生态系统崩溃到几乎相同的位置。极端的份额加上最强的注入偏差仍不能保证引导，其产生的主题偏移在衡量崩溃的标尺上只留下微弱的痕迹。决定速度的是谁提供池以及这些提供者被携带的容易程度：在保持所有份额固定的情况下，交换一个K=3生态系统的成员将五代漂移改变了2.8倍；每个成员易感性的份额加权指数解释了十九个臂之间速度差异的R^2=0.68；用人类文本替换一半的池大约将漂移减半而不改变其方向。在测试范围内，集中化既不决定崩溃的目的地也不决定其速度；速度取决于谁的文本填充了池。
英文摘要
More and more of the text future language models learn from is written by a few of today's models. If one supplier writes most of a shared corpus, does it pull the models trained on it toward its own writing, or change how fast they drift? We retrain eight open models from their base weights on a shared pool of each other's text for five generations, varying the part written by one model, Phi-2, from an equal share to 90%. The models drift together toward a style with fewer function words, and none starts repeating itself. No share of Phi-2 brings the other models closer to its text than the equal share does. We split each ecosystem's separation from the equal-share one into a delay along its route and a departure from that route, both counted beyond the difference between two equal-share runs. With Phi-2 at 90%, delay outweighs departure 72 to 28 and 64 to 36 in two runs, and the ecosystem falls 2.7 and 2.5 generations behind. With Phi-2 at half the pool the two parts are about equal. When SmolLM2 or Qwen3-1.7B writes half instead, the ecosystem slows less or not at all. The departure leans toward Phi-2 more as its share grows, but more than toward every other model only at 90%. Human text filling a quarter or half of the pool slows the models along the same route.
发表机构
- Shandong University（山东大学）
机构由 AI 辅助整理，请以论文原文为准。
