arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.09221cs.LG
收藏

超越测试时记忆:用于LLM推理的状态空间最优控制

Beyond Test-Time Memory: State-Space Optimal Control for LLM Reasoning

vita-group(vita组)

Peihao Wang, Shan Yang, Xijun Wang, Tesi Xiao, Xin Liu, Changlong Yu, Yu Lou, Pan Li, Zhangyang Wang, Ming Lin, René Vidal

AI总结:

提出测试时控制(TTC)层,通过有限时域LQR规划实现推理,作为适配器集成到预训练LLM中,在数学推理任务上提升高达27.8%的准确率。

摘要:

联想记忆长期以来支撑着序列模型的设计。除了回忆之外,人类通过预测未来状态和选择目标导向行动来进行推理,这是现代语言模型日益需要但并未原生编码的能力。虽然先前的工作使用强化学习或测试时训练,但规划仍然独立于模型架构。我们将推理形式化为最优控制,并引入测试时控制(TTC)层,该层在推理时对潜在状态执行有限时域LQR规划,在神经架构内表示价值函数,并利用它作为嵌套目标以实现预测前的规划。为了确保可扩展性,我们基于辛公式推导出一个硬件高效的LQR求解器,并将其实现为融合CUDA内核,从而以最小开销实现并行执行。作为适配器集成到预训练LLM中,TTC层在MATH-500上将数学推理性能提升高达27.8%,在AMC和AIME上实现2-3倍的Pass@8改进,证明将最优控制嵌入为架构组件为超越测试时训练的推理提供了有效且可扩展的机制。

英文摘要:

Associative memory has long underpinned the design of sequential models. Beyond recall, humans reason by projecting future states and selecting goal-directed actions, a capability that modern language models increasingly require but do not natively encode. While prior work uses reinforcement learning or test-time training, planning remains external to the model architecture. We formulate reasoning as optimal control and introduce the Test-Time Control (TTC) layer, which performs finite-horizon LQR planning over latent states at inference time, represents a value function within neural architectures, and leverages it as the nested objective to enable planning before prediction. To ensure scalability, we derive a hardware-efficient LQR solver based on a symplectic formulation and implement it as a fused CUDA kernel, enabling parallel execution with minimal overhead. Integrated as an adapter into pretrained LLMs, TTC layers improve mathematical reasoning performance by up to +27.8% on MATH-500 and 2-3x Pass@8 improvements on AMC and AIME, demonstrating that embedding optimal control as an architectural component provides an effective and scalable mechanism for reasoning beyond test-time training.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑