打断循环：周期性主体变化提升基础语言模型的惊讶度与关联度
Interrupting the Loop: Periodic Subject Changes Raise Judged Surprise and Connection in Base Language Models
浏览论文内容
中文总结 AI 辅助
本研究探究基础语言模型的长文本生成特性，发现周期性注入新主体的打断操作可提升模型生成文本的惊讶度与关联度，还提出了长文本生成的评估方案。
中文摘要 AI 辅助
无特定任务的基础语言模型产生的新颖性从何而来？大型语言模型（LLM）对长文本流的评判究竟能感知到什么？我们在三个基础模型的24种条件下，拆解了一个受认知启发的生成循环。该循环的大部分效果源于一个操作：每隔数百个标记就向文本流中注入一个新主体（即打断），而文本流的字面重复会被抑制（即习惯化）。我们仅以生成文本的窗口为评判对象，以前提为单位（n=10），并对评判者的可重复性进行了测量，同时与另一组评判者以及人类读者进行了对比。在该实验方案下，与仅习惯化的情况相比，打断使评判出的惊讶度提升了1.2至1.4个点，关联度提升了0.8个点。要求连续性的连接词会产生负面影响；单纯的段落分隔对新鲜文本没有可检测到的作用；重置上下文的效果至少与保留上下文相当；且在新前提下进行的预注册复制实验证实了这一主要对比。窗口评判者无法感知的三件事改变了本研究的第一个版本，我们认为它们具有普遍用途：评判者会将实验者注入的句子视为模型自身产生的；固定的注入句子轮换会使模型从评判者视野之外重放其早期片段，且评判者会将该重放评判为惊讶和关联（在150-300的周期下，65%-80%的打断后窗口会出现此情况）；局部增益无法组合：没有任何一个环节能生成整合的文档。显著性监视器、循环内评判者、跨打断记忆以及带有开启门控的评判者门控Review均无增益。在一个带有验证器的问题（在线装箱问题）上，打断使有效、独特的候选启发式方法增加了3至4倍，但并未提升最优方法的质量。我们报告了一种针对长文本生成的评估方案，以及对一种简单干预措施的受控表征，而非创造力的机制。
英文摘要
Where does the novelty a base language model produces with no task come from, and what can an LLM judge of a long stream actually see? We dismantle a cognitively inspired generation loop over 24 conditions on three base models. Most of its effect lives in one operation: a new subject injected every few hundred tokens (an interruption) into a stream whose literal repetition is damped (habituation). We judge windows of generated text only, with the premise as the unit (n=10) and a judge measured for repeatability, against a second judge family and against human readers. Under that protocol the interruption raises judged surprise by 1.2 to 1.4 points and connection by 0.8 over habituation alone. A connective that asks for continuity hurts; a bare paragraph break adds nothing detectable on fresh text; a reset context does at least as well as a kept one; and a pre-registered replication on new premises confirms the primary contrast. Three things the window judge could not see changed the first version of this study, and we think they are of general use. The judge scores the experimenter's injected sentence as the model's own. A fixed rotation of injected sentences makes the model replay its earlier segments from beyond the judge's horizon, and the judge scores the replay as surprise and connection (65-80% of post-interruption windows at periods 150-300). And the local gains do not compose: no arm produces an integrated document. The salience monitor, the in-loop judge, memory across interruptions and a judge-gated Review run with a gate that opens add nothing. On a problem with a verifier (online bin packing), the interruption multiplies valid, distinct candidate heuristics three- to fourfold without raising the quality of the best. We report an evaluation protocol for long generation and a controlled characterization of a simple intervention, not a mechanism of creativity.
