arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.08554cs.CLcs.AI
收藏

大语言模型文本生态系统中的漂移与选择

Drift and selection in LLM text ecosystems

Queen Mary University of London(伦敦玛丽女王大学)

Søren Riis

AI总结:

本文提出一个可解的数学框架,分析递归过程中的漂移与选择作用,揭示文本生态系统的稳定分布与选择过滤对结构的影响。

摘要:

公共文本记录——人类和AI系统现在学习的材料——日益受到自身输出的影响。生成的文本进入公共记录,后续智能体从中学习,这一过程循环往复。本文基于变阶n-gram智能体,开发了一个可解的数学框架,分别分析漂移和选择对公共语料库的影响。漂移是指未过滤的重复使用逐渐消除稀有形式,在无限语料库极限下精确刻画稳定分布。选择则涉及出版、排名和验证过滤进入记录的内容,结果取决于所选内容。当出版仅反映统计现状时,语料库收敛于浅层状态,进一步前瞻无益。当出版具有规范性——奖励质量、正确性或新颖性——深层结构得以维持,我们建立了由此产生的与浅层均衡偏离的最优上界。该框架明确了递归出版如何压缩公共文本,以及选择过滤如何维持更丰富的结构,对AI训练语料库设计有启示。

英文摘要:

The public text record -- the material from which both people and AI systems now learn -- is increasingly shaped by its own outputs. Generated text enters the public record, later agents learn from it, and the cycle repeats. Here we develop an exactly solvable mathematical framework for this recursive process, based on variable-order $n$-gram agents, and separate two forces acting on the public corpus. The first is drift: unfiltered reuse progressively removes rare forms, and in the infinite-corpus limit we characterise the stable distributions exactly. The second is selection: publication, ranking and verification filter what enters the record, and the outcome depends on what is selected. When publication merely reflects the statistical status quo, the corpus converges to a shallow state in which further lookahead brings no benefit. When publication is normative -- rewarding quality, correctness or novelty -- deeper structure persists, and we establish an optimal upper bound on the resulting divergence from shallow equilibria. The framework therefore identifies when recursive publication compresses public text and when selective filtering sustains richer structure, with implications for the design of AI training corpora.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑