发表机构
University College London（伦敦大学学院）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
LS-AR通过双通道架构将连续目标引导与离散令牌解码解耦，在长程检索和规划任务中显著提升性能与效率，并增强对提示注入的防御。
AI 中文摘要
标准自回归（AR）模型在单一共享的令牌序列中处理高级任务指令、状态历史和瞬时令牌。因此，它们缺乏将宏观目标与上下文噪声隔离所需的架构机制。为克服这一单通道限制，我们引入了潜在引导自回归（LS-AR），一种通过FiLM条件化将连续目标引导与离散令牌解码解耦的双通道架构。我们评估了用于在长程展开中持久保留宏观目标的静态目标编码器（P_0），以及用于在生成过程中进行循环潜在更新的动态状态跟踪器（P_t）。在超出上下文限制的长程检索（H=1024，W=500）中，LS-AR（静态）实现了100%的目标召回率，而参数匹配的基线模型则完全失效（0%），同时吞吐量提高了约35%，峰值显存降低了52.8%。在强制扰动（k=1）下的Blocksworld规划中，LS-AR（动态）保持了89.0%的完成率，而基线为71.0%，尽管零样本实体扩展（N -> N+1）暴露了单向量容量限制（0%）。最后，双通道权威分析表明，文本目标丢弃建立了潜在主导控制，提供了针对文本提示注入的结构性防御，同时引入了潜在向量攻击面。
英文摘要
Standard autoregressive (AR) models process high-level task instructions, state history, and transient tokens within a single shared sequence of tokens. Consequently, they lack the architectural mechanisms needed to isolate macro-objectives from context noise. To overcome this single-channel limitation, we introduce Latent-Steered Autoregressive (LS-AR), a dual-channel architecture that decouples continuous goal steering from discrete token decoding via FiLM conditioning. We evaluate a Static Goal Encoder (P_0) for persistent macro-objective retention across long rollouts and a Dynamic State Tracker (P_t) for recurrent latent updates during generation. On long-horizon retrieval past context limits (H=1024, W=500), LS-AR (Static) achieves 100% target recall where parameter-matched baselines collapse (0%), while increasing throughput by ~35% and cutting peak VRAM by 52.8%. In Blocksworld planning under forced perturbations (k=1), LS-AR (Dynamic) sustains an 89.0% completion rate vs. 71.0% for the baseline, though zero-shot entity scaling (N -> N+1) exposes single-vector capacity limits (0%). Finally, dual-channel authority analysis shows that text goal dropout establishes latent-dominant control, offering structural defence against text prompt injection while introducing a latent vector attack surface.
CommentsAccepted at the NeurIPS 2026 Workshop: Long-Context Foundation Models
