推理模型生成思想社会
Reasoning Models Generate Societies of Thought
AI总结：
推理模型通过模拟多代理互动生成思想社会，提升推理准确性与多样性，揭示了社会结构对问题解决的促进作用。
摘要：
大型语言模型在多个领域中取得了显著的能力，但复杂推理背后的机制仍然不清楚。最近的推理模型在复杂认知任务上优于相应的指令调优模型，这归因于通过更长的思考链进行扩展计算。在这里，我们表明，增强的推理并不来自扩展计算本身，而是来自模拟多代理式互动——一个思想社会——这使得内部认知视角之间能够产生多样化和辩论，这些视角具有不同的个性特征和领域专业知识。通过定量分析和机械可解释性方法应用于推理轨迹，我们发现，推理模型如DeepSeek-R1和QwQ-32B比指令调优模型具有更大的视角多样性，激活了在推理过程中更广泛的异质性个性和专业知识相关特征之间的冲突。这种多代理结构在对话行为中表现出来，包括问答、视角转换和冲突观点的调和，并在社会情感角色中体现出来，这些角色特征了尖锐的来回对话，共同导致了推理任务中的准确性优势。受控强化学习实验表明，基础模型在仅被奖励推理准确性时会增加对话行为，而通过对话支架微调的模型在基础模型上加速了推理改进。这些发现表明，思想的社会组织能够有效探索解决方案空间。我们建议，推理模型建立了一种计算上的类比，即人类群体中的集体智能，其中多样性在系统化结构下能够实现优越的问题解决能力，这提出了新的机会，使代理组织能够利用群体智慧。
英文摘要：
Large language models have achieved remarkable capabilities across domains, yet mechanisms underlying sophisticated reasoning remain elusive. Recent reasoning models outperform comparable instruction-tuned models on complex cognitive tasks, attributed to extended computation through longer chains of thought. Here we show that enhanced reasoning emerges not from extended computation alone, but from simulating multi-agent-like interactions -- a society of thought -- which enables diversification and debate among internal cognitive perspectives characterized by distinct personality traits and domain expertise. Through quantitative analysis and mechanistic interpretability methods applied to reasoning traces, we find that reasoning models like DeepSeek-R1 and QwQ-32B exhibit much greater perspective diversity than instruction-tuned models, activating broader conflict between heterogeneous personality- and expertise-related features during reasoning. This multi-agent structure manifests in conversational behaviors, including question-answering, perspective shifts, and the reconciliation of conflicting views, and in socio-emotional roles that characterize sharp back-and-forth conversations, together accounting for the accuracy advantage in reasoning tasks. Controlled reinforcement learning experiments reveal that base models increase conversational behaviors when rewarded solely for reasoning accuracy, and fine-tuning models with conversational scaffolding accelerates reasoning improvement over base models. These findings indicate that the social organization of thought enables effective exploration of solution spaces. We suggest that reasoning models establish a computational parallel to collective intelligence in human groups, where diversity enables superior problem-solving when systematically structured, which suggests new opportunities for agent organization to harness the wisdom of crowds.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
