能量引导递归模型

中文摘要 AI 辅助

递归推理模型通过反复更新小型神经网络的潜在状态来解决结构化问题。然而，其测试时的扩展缺乏有原则的推理机制。本文开发了能量引导递归模型（ERM），它基于显式霍普菲尔德能量引入了内在选择原则。ERM利用有效局部或全局结构的霍普菲尔德型记忆来定义候选轨迹的选择器。所得能量与并行回火等基于能量的技术无缝集成，提高采样效率和排序。在特定递归步数和候选数下，ERM在数独、铅笔谜题基准和迷宫问题上达到最优解，优于近期其他模型。结果表明将显式能量函数纳入递归推理为更有效的推理提供了原则性途径。

英文摘要

Recursive models show promise on reasoning and language tasks, yet their test-time scaling lacks a principled criterion for selecting trajectories or determining recurrent depth. We introduce \textbf{Energy-guided Recursive Model (ERM)}, which uses Hopfield-type memories of valid local and global structures to assign intrinsic energies to candidate trajectories. These energies guide candidate selection and suggest an effective range of recurrent depths, implying that deeper recurrence does not necessarily improve reasoning accuracy. They also enable sampling methods such as parallel tempering to improve exploration. For reasoning tasks, ERM achieves optimal solutions on Sudoku ($98.97\%$), Pencil Puzzle Bench (PPBench, $88.04\%$) and Maze ($99.30\%$), reaching the best accuracy in recursive modeling. On language modeling, ERM reduces RedPajama-V2 perplexity by $1.74\%$ with marginal inference overhead. The results support energy guidance as a practical framework for improving test-time scaling in recursive models.

发表机构

• Institute of Fundamental and Frontier Sciences, University of Electronic Science and Technology of China（电子科技大学基础与前沿研究院）
• School of Physics, University of Electronic Science and Technology of China（电子科技大学物理学院）
• Key Laboratory of Quantum Physics and Photonic Quantum Information, Ministry of Education, University of Electronic Science and Technology of China（电子科技大学量子物理与光子量子信息教育部重点实验室）
• Non-classical Information Science Basic Discipline Research Center of Sichuan Province, University of Electronic Science and Technology of China（四川省非经典信息科学基础学科研究中心）

机构由 AI 辅助整理，请以论文原文为准。