arXivDaily arXiv每日学术速递 周一至周五更新
北美退伍军人节(Veterans Day),12 日无更新
arXiv 2510.04871cs.LGcs.AI
收藏

少即是多:用微型网络进行递归推理

Less is More: Recursive Reasoning with Tiny Networks

Alexia Jolicoeur-Martineau

AI总结:

针对HRM潜力大但可能非最优的问题,提出仅含2层、7M参数的微型递归模型TRM,以更简单的递归推理实现更强泛化,在ARC-AGI等谜题任务上准确率超多数大模型,参数量不到其0.01%。

摘要:

分层推理模型(Hierarchical Reasoning Model,HRM)是一种采用两个以不同频率递归的小型神经网络的新方法。这种受生物启发的方法在数独、迷宫和ARC-AGI等困难谜题任务上优于大语言模型(Large Language models,LLMs),且仅用小模型(27M参数)在小数据(约1000个样本)上训练。HRM在利用小型网络解决困难问题方面极具潜力,但目前尚未被充分理解,且可能并非最优。我们提出微型递归模型(Tiny Recursive Model,TRM),这是一种更简单的递归推理方法,仅使用一个仅含2层的微型网络,泛化能力显著高于HRM。TRM仅含7M参数,在ARC-AGI-1上取得45%的测试准确率,在ARC-AGI-2上取得8%的测试准确率,优于大多数LLMs(如Deepseek R1、o3-mini、Gemini 2.5 Pro),而参数量不到这些模型的0.01%。

英文摘要:

Hierarchical Reasoning Model (HRM) is a novel approach using two small neural networks recursing at different frequencies. This biologically inspired method beats Large Language models (LLMs) on hard puzzle tasks such as Sudoku, Maze, and ARC-AGI while trained with small models (27M parameters) on small data (around 1000 examples). HRM holds great promise for solving hard problems with small networks, but it is not yet well understood and may be suboptimal. We propose Tiny Recursive Model (TRM), a much simpler recursive reasoning approach that achieves significantly higher generalization than HRM, while using a single tiny network with only 2 layers. With only 7M parameters, TRM obtains 45% test-accuracy on ARC-AGI-1 and 8% on ARC-AGI-2, higher than most LLMs (e.g., Deepseek R1, o3-mini, Gemini 2.5 Pro) with less than 0.01% of the parameters.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑