速度即信心
Speed is Confidence
AI总结:
通过将神经元放电视为信心,该研究在Sudoku-Extreme上实现了97%的准确率,通过仅训练成本和并行潜在状态提升性能。
摘要:
生物神经系统必须快速但受到能量限制。进化解决方案:行动于第一个信号。赢家通吃电路和时间到第一个脉冲编码隐含地将神经元放电视为信心的表达。我们将这一原则应用于Tiny Recursive Models(TRM)的集合中[Jolicoeur-Martineau et al., 2025]。在Sudoku-Extreme上,先停止选择实现了97%的准确率,而概率平均为91%——同时仅需10倍更少的推理步骤。一个基线模型实现了85.5%±1.3%的准确率。我们可以将这一原则作为仅训练的成本内部化吗?是的:通过维护K=4个并行的潜在状态但仅通过最低损失的“赢家”反向传播,我们实现了96.9%±0.6%的准确率——在1倍推理成本下达到集合性能,其方差不到基线的一半。一个关键的诊断:89%的基线失败是选择问题,揭示了99%的准确率上限。正如在自然界中,这项工作也受到资源限制:所有实验都使用了一个RTX 5090。经过修改的SwiGLU[Shazeer, 2020]使Muon[Jordan et al., 2024]和高LR成为可能,使基线训练在48分钟内完成,完整WTA(K=4)在消费者硬件上6小时内完成。
英文摘要:
Biological neural systems must be fast but are energy-constrained. Evolution's solution: act on the first signal. Winner-take-all circuits and time-to-first-spike coding implicitly treat when a neuron fires as an expression of confidence. We apply this principle to ensembles of Tiny Recursive Models (TRM) [Jolicoeur-Martineau et al., 2025]. On Sudoku-Extreme, halt-first selection achieves 97% accuracy vs. 91% for probability averaging -- while requiring 10x fewer reasoning steps. A single baseline model achieves 85.5% +/- 1.3%. Can we internalize this as a training-only cost? Yes: by maintaining K=4 parallel latent states but backpropping only through the lowest-loss "winner," we achieve 96.9% +/- 0.6% accuracy -- matching ensemble performance at 1x inference cost, with less than half the variance of the baseline. A key diagnostic: 89% of baseline failures are selection problems, revealing a 99% accuracy ceiling. As in nature, this work was also resource constrained: all experiments used a single RTX 5090. A modified SwiGLU [Shazeer, 2020] made Muon [Jordan et al., 2024] and high LR viable, enabling baseline training in 48 minutes and full WTA (K=4) in 6 hours on consumer hardware.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。