大语言模型中的叙事前瞻:事件级可解码性、预测优势与调控
Narrative Lookahead in LLMs: Event-Level Decodability, Predictive Advantage, and Steering
AI总结:
本研究以事件为单位考察大语言模型的叙事前瞻,发现其可解码至少4个后续事件,且基于探测的调控方向能改变目标事件类型比例,为叙事预测与干预提供了新方法。
摘要:
语言模型在生成文本前是否携带关于叙事事件的信息,且能前瞻多远?本研究通过显著的状态变化事件而非 token 或响应级属性来考察叙事前瞻。我们从四个开放权重大语言模型(LLM)生成的2000篇故事中提取了23355个经 LLM 标注的事件,沿三个非互斥的变化维度标注:主体内部、主体之间及故事世界的变化,并对部分样本进行人工验证。事件分割与独立于情节的标签使我们能以事件而非 token 为单位的跨度对不同故事进行比较。在事件结尾处,我们探测隐藏状态以获取即将发生的事件类型,并使用匹配的前缀将预测结果与仅基于文本的预测器进行比较。在四个模型中,未来事件类型的可解码性高于流行度基线,至少可前瞻4个后续事件,但探测模型仅对下一个事件的表现显著优于主要的仅文本预测器。在使用 OLMo-3.1-32B 进行的探索性调控中,源自探测模型的方向在较高强度下可按预期方向改变目标事件类型的比例。我们的事件级方法将未来叙事发展的预测与重塑后续事件构成的干预措施联系起来。
英文摘要:
Do language models carry information about narrative events before writing them, and how far ahead? We examine narrative lookahead through salient, state-changing events rather than tokens or response-level attributes. We extract 23,355 LLM-annotated events from 2,000 stories generated by four open-weight LLMs, labeling them along three nonexclusive dimensions of change: within agents, between agents, and in the story world, with human validation on a subset. Event segmentation and plot-independent labels enable comparisons across stories at horizons measured in events rather than in tokens. At event ends, we probe hidden states for upcoming event types and compare predictions with text-only predictors using matched prefixes. Across four models, future event types are decodable above prevalence baselines for at least four subsequent events, but probes significantly outperform both primary text-only predictors only for the next event. In exploratory steering with OLMo-3.1-32B, a probe-derived direction shifts the proportion of the target event type in both intended directions at higher strengths. Our event-level approach links the prediction of future narrative developments to interventions that reshape subsequent event composition.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。