基于注意力机制的高维非线性自回归模型：训练、对齐与推理
LLMs as High-Dimensional Nonlinear Autoregressive Models with Attention: Training, Alignment and Inference
AI总结：
本文将LLMs表述为具有注意力依赖的高维非线性自回归模型，探讨了训练、对齐与推理的原理及方法。
摘要：
基于变换器架构的大语言模型（LLMs）通常通过一系列架构组件和训练流程来描述，这掩盖了其底层的计算结构。本文综述文章为寻求明确、方程级描述LLM训练、对齐和生成的学者提供了一个简洁的数学参考。我们把LLMs表述为具有注意力依赖的高维非线性自回归模型。该框架涵盖了通过下个token预测进行预训练，对齐方法如人类反馈强化学习（RLHF）、直接偏好优化（DPO）、拒绝采样微调（RSFT）以及基于可验证奖励的强化学习（RLVR），以及推理时的自回归生成。自注意力自然地出现为重复的双线性-softmax-线性组合，产生高度表达的序列模型。这种表述使能够对齐诱导行为（包括奉承）进行原理性分析，以及推理时的现象（如幻觉、上下文学习、思维链提示和检索增强生成），并扩展了持续学习等扩展内容，同时为解释和进一步的理论发展提供了一个简洁的参考。
英文摘要：
Large language models (LLMs) based on transformer architectures are typically described through collections of architectural components and training procedures, obscuring their underlying computational structure. This review article provides a concise mathematical reference for researchers seeking an explicit, equation-level description of LLM training, alignment, and generation. We formulate LLMs as high-dimensional nonlinear autoregressive models with attention-based dependencies. The framework encompasses pretraining via next-token prediction, alignment methods such as reinforcement learning from human feedback (RLHF), direct preference optimization (DPO), rejection sampling fine-tuning (RSFT), and reinforcement learning from verifiable rewards (RLVR), as well as autoregressive generation during inference. Self-attention emerges naturally as a repeated bilinear--softmax--linear composition, yielding highly expressive sequence models. This formulation enables principled analysis of alignment-induced behaviors (including sycophancy), inference-time phenomena (such as hallucination, in-context learning, chain-of-thought prompting, and retrieval-augmented generation), and extensions like continual learning, while serving as a concise reference for interpretation and further theoretical development.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
