基于神经元的大模型中神经元交互动力学多重分形分析
Neuron-based Multifractal Analysis of Neuron Interaction Dynamics in Large Models
AI总结：
针对大型模型涌现能力内部机制缺乏定量分析的问题，提出基于神经元的多重分形分析框架NeuroMFA，通过结构度量代理涌现能力，实验验证其有效性。
摘要：
近年来，大型模型的能力日益受到关注，尤其是在处理小规模模型无法完成的复杂任务方面。值得注意的是，大型语言模型（LLMs）展现出了“智能”能力，如复杂推理和抽象语言理解，反映了类似认知的行为。然而，当前关于大型模型涌现能力的研究主要集中在模型性能与规模之间的关系上，对驱动这些涌现能力的内部结构和机制的系统性定量分析仍存在显著空白。受神经科学中关于脑网络结构和自组织研究的启发，我们提出了（i）大型模型的通用网络表示，（ii）一种新的分析框架，称为基于神经元的多重分形分析（NeuroMFA），用于结构分析，以及（iii）一种新颖的基于结构的度量，作为大型模型涌现能力的代理指标。通过将结构特征与大型模型的能力联系起来，NeuroMFA为分析大型模型中的涌现现象提供了一个定量框架。我们的实验表明，所提出的方法能够全面衡量网络演化的异质性和组织性，为研究大型模型中的涌现能力提供了理论基础和新视角。
英文摘要：
In recent years, there has been increasing attention on the capabilities of large models, particularly in handling complex tasks that small-scale models are unable to perform. Notably, large language models (LLMs) have demonstrated ``intelligent'' abilities such as complex reasoning and abstract language comprehension, reflecting cognitive-like behaviors. However, current research on emergent abilities in large models predominantly focuses on the relationship between model performance and size, leaving a significant gap in the systematic quantitative analysis of the internal structures and mechanisms driving these emergent abilities. Drawing inspiration from neuroscience research on brain network structure and self-organization, we propose (i) a general network representation of large models, (ii) a new analytical framework, called Neuron-based Multifractal Analysis (NeuroMFA), for structural analysis, and (iii) a novel structure-based metric as a proxy for emergent abilities of large models. By linking structural features to the capabilities of large models, NeuroMFA provides a quantitative framework for analyzing emergent phenomena in large models. Our experiments show that the proposed method yields a comprehensive measure of network's evolving heterogeneity and organization, offering theoretical foundations and a new perspective for investigating emergent abilities in large models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
