arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2506.13331cs.LG
收藏

认知推理的混合:类脑的专业化模块推理

Mixture of Cognitive Reasoners: Modular Reasoning with Brain-Like Specialization

EPFL(瑞士联邦理工学院洛桑);Brain and Cognitive Sciences at MIT(麻省理工学院脑与认知科学系);Kempner Institute at Harvard University(哈佛大学肯普纳研究所)

Badr AlKhamissi, C. Nicolò De Sabbata, Greta Tuckute, Zeming Chen, Martin Schrimpf, Antoine Bosselut

AI总结:

本文提出MiCRo模型,通过类脑专业化模块实现认知行为,提供可解释的推理模块,支持动态路由并优于基线模型。

摘要:

人类认知行为源于专门化脑网络的交互,如语言、逻辑和社会推理。受此启发,我们提出MiCRo:一种模块化、基于Transformer的架构,通过课程学习诱导专家功能专业化。具体而言,我们将预训练语言模型的层划分为四个专家模块,对应人类大脑中已研究的认知网络。MiCRo相比标准语言模型有三个优势:(1)专业化专家可解释且因果意义明显;(2)推理时可通过路由令牌动态控制行为;(3)在机器学习推理基准(如GSM8K、BBH)和对齐人类行为(CogBench)上表现更优或匹配基线,同时保持可解释性。总体而言,基于认知的功能专业化产生更人样且更可解释的模型。

英文摘要:

Human cognitive behavior arises from the interaction of specialized brain networks dedicated to distinct functions, such as language, logic, and social reasoning. Inspired by this organization, we propose Mixture of Cognitive Reasoners (MiCRo): a modular, transformer-based architecture post-trained with a curriculum that induces functional specialization across experts. Concretely, we partition the layers of a pretrained language model into four expert modules aligned with well-studied cognitive networks in the human brain. MiCRo offers three key advantages over standard language models. (1) The specialized experts are interpretable and causally meaningful -- ablating a module causes substantial drops on benchmarks requiring its specialized domain. (2) MiCRo's behavior can be dynamically steered at inference time by routing tokens to particular experts (e.g., favoring social over logical reasoning), enabling fine-grained control over outputs. (3) MiCRo outperforms or matches comparable baselines on both machine-learning reasoning benchmarks (e.g., GSM8K, BBH) and alignment to human behavior (CogBench), while maintaining interpretability. Taken together, cognitively grounded functional specialization yields models that are both more human-like and more human-interpretable.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑