置换等变流匹配用于免对齐神经权重生成
Permutation-Equivariant Flow Matching for Alignment-Free Neural Weight Generation
浏览论文内容
中文总结 AI 辅助
本文提出用置换等变图元网络参数化流匹配速度场，实现无需对齐的神经权重生成，能再现独立训练网络的统计特性并泛化到新架构。
中文摘要 AI 辅助
一个训练好的神经网络可以用高维参数向量表示。学习这些向量的分布能够生成跨任务和架构的新模型。一个核心挑战是置换对称性：置换隐藏神经元可以产生表示相同函数但相距遥远的参数向量。这引入了生成模型在从训练好的网络中学习时必须考虑的变异性。现有方法通常使用从共同基础模型派生的网络或代价高昂的近似神经元对齐来解决这一问题。我们转而使用置换等变图元网络参数化流匹配速度场，从而能够直接从独立训练的网络中学习，无需对齐。大量实验表明，我们的方法紧密再现了独立训练集合的准确率、功能相似性和权重相似性的联合统计，提供了超越检查点记忆的生成证据。一个条件模型还能在异构架构上生成任务特定网络，并泛化到未见过的隐藏宽度配置。在表格域偏移任务上，中间条件化产生的单个网络在两个域上的性能与logit集成相当。综合来看，我们的结果展示了置换等变性如何使从多样化的独立训练网络集合中学习成为可能，而无需置换对齐。
英文摘要
A trained neural network can be represented by a parameter vector in high dimensions. Learning distributions over these vectors enables the generation of new models across various tasks and architectures. A central challenge is permutation symmetry: permuting hidden neurons can produce distant parameter vectors representing the same function. This introduces variations that a generative model must account for when learning from trained networks. Existing methods typically address this using networks derived from a common base model or costly approximate neuron alignment. We instead parameterize a flow-matching velocity field with a permutation-equivariant Graph Meta Network, enabling direct learning from independently trained networks without alignment. Extensive experiments show that our method closely reproduces the joint statistics of accuracy, functional similarity, and weight similarity of independently trained collections, providing evidence of generation beyond checkpoint memorization. A single conditional model also generates task-specific networks on heterogeneous architectures and generalizes to unseen hidden-width configurations. On a tabular domain-shift task, intermediate conditioning produces individual networks with performance comparable to logit ensembles across both domains. Taken together, our results show how permutation equivariance enables learning from diverse collections of independently trained networks without permutation alignment.
