最近关于递归推理模型如TRM的研究表明,微小网络(7M参数)可通过隐含递归在抽象推理任务中实现强性能,通过隐藏表征空间中的迭代细化实现。这引发了关于操作符选择的自然问题:Mamba-2的状态空间递归本身是一种迭代细化形式,使其成为递归推理的自然候选者——但将Mamba-2引入递归框架是否能保持推理能力?我们通过将TRM中的Transformer块替换为Mamba-2混合操作符进行研究,保持参数量一致(6.83M vs 6.86M参数)。在ARC-AGI-1上,混合模型在pass@2(官方指标)上提升+2.0%(45.88% vs 43.88%),并在更高K值时持续表现更优(pass@100时+4.75%),同时保持pass@1性能一致。这表明模型生成正确解决方案的可靠性提升,同时保持top-1选择率相似。结果验证了Mamba-2混合操作符在递归框架内保持推理能力,确立了基于SSM的操作符作为递归操作符设计空间中的可行候选,并为理解递归推理的最佳混合策略迈出第一步。
英文摘要:
Recent work on recursive reasoning models like TRM demonstrates that tiny networks (7M parameters) can achieve strong performance on abstract reasoning tasks through latent recursion -- iterative refinement in hidden representation space without emitting intermediate tokens. This raises a natural question about operator choice: Mamba-2's state space recurrence is itself a form of iterative refinement, making it a natural candidate for recursive reasoning -- but does introducing Mamba-2 into the recursive scaffold preserve reasoning capability? We investigate this by replacing the Transformer blocks in TRM with Mamba-2 hybrid operators while maintaining parameter parity (6.83M vs 6.86M parameters). On ARC-AGI-1, we find that the hybrid improves pass@2 (the official metric) by +2.0\% (45.88\% vs 43.88\%) and consistently outperforms at higher K values (+4.75\% at pass@100), whilst maintaining pass@1 parity. This suggests improved candidate coverage -- the model generates correct solutions more reliably -- with similar top-1 selection. Our results validate that Mamba-2 hybrid operators preserve reasoning capability within the recursive scaffold, establishing SSM-based operators as viable candidates in the recursive operator design space and taking a first step towards understanding the best mixing strategies for recursive reasoning.
Reliability-Calibrated Edge-IoT Early Fault Warning for Rotating Machinery with a Physics-Guided Tiny-Mamba Transformer
Great Bay University(大亚湾大学);Huizhou University(惠州大学);National University of Singapore(国立新加坡大学);Shenzhen University(深圳大学);James Cook University(詹姆斯库克大学);Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州))
Attention-Mamba: A Mamba-Enhanced Multi-Scale Parallel Inference Network for Medical Image Segmentation
School of Astronautics, Northwestern Polytechnical University(航天学院,西北工业大学);Department of Electronics and Telecommunications, Politecnico di Torino(电信系,托斯卡纳理工学院);Beijing Aerospace Automatic Control Research Institute(北京航天自动控制研究所);Xi'an University of Posts and Telecommunications(西安邮电大学);Department of Oncology, University of Turin(肿瘤科,都灵大学);Candiolo Cancer Institute, FPO-IRCCS(坎迪奥利癌症研究所,FPO-IRCCS)
Department of Computer Science, Technical University of Darmstadt(德累斯顿技术大学计算机科学系);Department of Computer Science, The University of Texas at Dallas(德克萨斯大学达拉斯分校计算机科学系);Hessian Center for Artificial Intelligence (hessian.ai)(黑森人工智能中心);German Research Center for AI (DFKI)(德国人工智能研究中心)
HARMamba: Efficient and Lightweight Wearable Sensor Human Activity Recognition Based on Bidirectional Mamba
School of Computer Science, University of South China(南华大学计算机学院);School of Computer Science and Technology, Dalian University of Technology(大连理工大学计算机科学与技术学院);Department of Computer & Communication Engineering, University of Science and Technology Beijing(北京科技大学计算机与通信工程学院);School of Computing, Ulster University(阿尔斯特大学计算学院)
PRefLexOR: Preference-based Recursive Language Modeling for Exploratory Optimization of Reasoning and Agentic Thinking
Center for Computational Science and Engineering(计算科学与工程中心);Schmarzman College of Computing(施马茨曼计算学院);Laboratory for Atomistic and Molecular Mechanics (LAMM)(原子与分子力学实验室);Massachusetts Institute of Technology(麻省理工学院)
Dynamical Systems Theory Behind a Hierarchical Reasoning Model
Department of Radiophysics, University of Nizhny Novgorod(尼日尼诺夫戈罗德大学无线电物理系);Huawei Nizhny Novgorod Research Center(华为尼日尼诺夫戈罗德研究中心);Skolkovo Institute of Science and Technology(斯克洛科夫科学与技术研究院)
University of Texas at Austin(德克萨斯大学奥斯汀分校);Tufts University(塔夫茨大学);Zyphra Technologies(Zyphra 科技公司);Austin Community College(奥斯汀社区学院);SpaceXAI(SpaceXAI 公司)