单纯形上的推理：几何不动点模型

中文摘要 AI 辅助

循环推理器通过迭代权重共享映射来消耗测试时计算，但当该映射位于无约束的潜在空间时，较小的残差并不意味着状态是不动点。我们提出几何不动点推理（GFPR），其中迭代状态本身就是预测：一个在单纯形乘积上的分类信念场，其argmax在每一步都是答案。由于状态是信念，任务结构可以通过紧凑的凸松弛来施加，既可以作为结构化读出，也可以直接用于循环状态；在后一种情况下，更新仍然是连续的自身映射，因此对于任何参数都存在不动点。在大约7M参数下，GFPR在Sudoku-Extreme上达到95.1%的精确匹配，在Maze-Hard上达到92.0%，在S_5长度128上达到100%的序列准确率，高于相同规模下已发表的FPRM数字。相同的更新还在FineWeb-Edu上训练了一个201M的语言模型，其中每个位置都是词汇表上的分布；经过24次Picard迭代，它在四个零样本多项选择任务上高于GPT-2 small，在ARC-Easy上高于GPT-2 medium。

英文摘要

Looped reasoners spend test-time compute by iterating a weight-tied map, but a small residual does not mean the state is a fixed point when that map lives in unconstrained latent space. We propose Geometric Fixed-Point Reasoning (GFPR), in which the iterated state is the prediction itself: a field of categorical beliefs on a product of simplices, whose argmax is the answer at every step. Because the state is a belief, task structure can be imposed through compact convex relaxations, either as structured readouts or directly in the recurrent state; in the latter case the update remains a continuous self-map, so a fixed point exists for any parameters. At about 7M parameters, GFPR reaches 95.1% exact match on Sudoku-Extreme, 92.0% on Maze-Hard, and 100% sequence accuracy on S_5 length 128, above the published FPRM numbers at the same scale. The same update also trains a 201M language model on FineWeb-Edu in which each site is a distribution over the vocabulary; with 24 Picard steps it is above GPT-2 small on four zero-shot multiple-choice tasks and above GPT-2 medium on ARC-Easy.

发表机构

• Applied AI Institute（应用人工智能研究所）
• LigandPro
• HSE University（高等经济大学）

机构由 AI 辅助整理，请以论文原文为准。