前馈网络与卷积网络之间的继承关系：通过模型投影
Inheritance Between Feedforward and Convolutional Networks via Model Projection
AI总结：
提出模型继承概念，证明广义前馈网络是广义卷积网络的子集，并通过模型投影实现反向继承，用于参数高效的迁移学习。
摘要：
神经网络技术通常通过类比在不同架构家族之间转移，但这种转移仅在技术所需假设被保留时才有效。我们将这一思想引入为模型类之间的继承。使用统一的节点级框架和张量值激活，我们证明广义前馈网络（GFFN）是广义卷积网络（GCNN）的严格子集，因此GCNN的性质直接转移到GFFN。反向方向并非自动：标准CNN节点使用空间核，而FFN节点对每个输入贡献使用一个标量权重。我们引入模型投影来恢复受限的反向继承路径。投影冻结每个卷积输入通道子函数，并为每个输入-输出通道贡献学习一个标量系数，使投影后的CNN节点具有标量加权输入重组的GFFN风格可训练结构。这种继承结构自然导致参数高效的迁移学习。在多个ImageNet预训练CNN骨干网络和下游图像分类数据集上，模型投影与标准和PEFT基线竞争，并为后续全微调提供有效的初始化。
英文摘要：
Neural-network techniques are often transferred across architecture families by analogy, but such transfer is valid only when the assumptions required by a technique are preserved. We introduce this idea as inheritance between model classes. Using a unified node-level framework with tensor-valued activations, we prove that generalized feedforward networks (GFFNs) form a strict subset of generalized convolutional networks (GCNNs), so GCNN properties transfer directly to GFFNs. The reverse direction is not automatic: standard CNN nodes use spatial kernels, while FFN nodes use one scalar weight per input contribution. We introduce model projection to recover a restricted reverse inheritance path. Projection freezes each convolutional input-channel sub-function and learns one scalar coefficient for each input-output channel contribution, giving projected CNN nodes the GFFN-style trainable structure of scalar-weighted input recombination. This inherited structure leads naturally to parameter-efficient transfer learning. Across multiple ImageNet-pretrained CNN backbones and downstream image-classification datasets, model projection is competitive with standard and PEFT baselines and provides an effective initialization for subsequent full fine-tuning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
