AI 中文总结
本研究通过17个现成模型的短窗口argmax不动点普查，发现重复生成分类跨模型稳定，但语料库内仅一个家族呈漏斗型，无法证明分裂与语料库无关。
AI 中文摘要
关于神经文本退化的两种解释并存。一种将原因归于训练数据——语料库中的重复导致输出中的重复，这一点通过对按重复度排序的数据进行训练而确立；另一种则归于训练好的网络，即复制电路和重复特征。这两种解释均未在广泛的预训练模型群体中得到仲裁：因果性研究训练的是其自身的模型。我们报告了一种以不同货币进行的观测性测量：模型自身短窗口argmax映射的不动点结构，该结构从17个现成模型上的96个随机双词元起始点普查而来，始终无提示——一篇配套论文表明，九个词元的条件化可将此读数移动至其大部分范围。四路分类在17个模型中的17个上跨普查种子稳定。三个示例。在固定语料库（The Pile）、固定规模和固定领域下，该分类并非确定：跨两个规模匹配的层级，pythia是漏斗型，而RWKV、Mamba和第二个Transformer家族则不是，且两者在跨越一个数量级的规模上均保持其分类。该阶梯中七个模型中的六个达到相同的终点词元，而那些最强烈集中于该词元的模型却从不驻留于此——变化的不是轨迹去向，而是目的地是否自我延续。去重后的Pythia套件不改变该分类。而针对此现象提出的语料侧流入项，在控制频率后，在英语和其他三种语言中并不选择我们的终点。这是观测性的，无法反驳训练干预。漏斗型很常见：十七个模型中的八个、七个家族、五个语料库——因此限制不在于该现象是单一模型的怪癖，而在于在训练数据可保持固定的唯一语料库中，仅有一个可用家族呈现漏斗型；该子集无法显示该分裂是语料库无关的。
英文摘要
Two accounts of neural text degeneration coexist. One locates the cause in the training data -- repetition in the corpus produces repetition in the output, established by training on repetition-sorted data -- the other in the trained network, in copying circuits and repetition features. Neither has been arbitrated across a broad cohort of pretrained models: the causal work trains its own. We report an observational measurement in a different currency: the fixed-point structure of a model's own short-window argmax map, censused from 96 random two-token starts over 17 off-the-shelf models, always unprompted -- a companion paper shows nine tokens of conditioning move this readout across most of its range. The four-way class is stable across census seeds on 17 of 17. Three exhibits. At fixed corpus (The Pile), fixed scale and that fixed domain, the class is not determined: across two size-matched tiers, pythia is a funnel while RWKV, Mamba and a second transformer family are not, and both hold their class across an order of magnitude of scale. Six of seven models in that ladder reach the same endpoint token, and those concentrating on it most strongly are among those that never stay there -- what varies is not where trajectories go but whether the destination self-continues. The deduplicated Pythia suite does not change the class. And the corpus-side inflow term proposed for this phenomenon does not select our endpoints once frequency is controlled, in English and three other languages. This is observational and cannot refute a training intervention. Funnels are common: eight of seventeen models, seven families, five corpora -- so the limit is not that the phenomenon is one model's peculiarity, but that within the one corpus where training data can be held fixed only one available family funnels; that subset cannot show the split is corpus-independent.
Comments9 pages, 2 tables. Companion to arXiv:2608.21315 and arXiv:2608.10986. Code, per-run results, and the findings ledger: https://github.com/nicoveraz/token-lattice-ca (archived: https://doi.org/10.5281/zenodo.21880472)
