产品交互:深度学习架构的一种代数形式化
Product Interaction: An Algebraic Formalism for Deep Learning Architectures
AI总结:
本文提出了一种代数形式化方法,通过产品交互构建深度学习架构,统一了线性、二次及更高阶的交互方式,适用于卷积、等变网络及注意力机制等模型。
摘要:
在本文中,我们介绍了产品交互,一种代数形式化方法,其中神经网络层由在适当代数上定义的乘法算子的组合构成。产品交互提供了一种系统的方法来生成和组织代数表达式,通过增加交互阶数。我们的核心观察是,现代神经网络中的代数表达式可以统一地用线性、二次和更高阶的产品交互来构建。卷积和等变网络作为对称约束下的线性产品交互,而注意力和Mamba则对应于更高阶的产品交互。
英文摘要:
Modern neural architectures are largely developed as distinct empirical constructions, despite recurring principles across convolutional, attention, state-space, and equivariant models. We introduce Product Interactions, an algebraic formalism that unifies these architectures under a common construction principle and turns their design into a systematic procedure. The core shift is representing data as algebraic elements and treating multiplication, rather than linear transformation alone, as a primitive architectural operation. Architectures are generated by composing multiplication operators, with input dependence organized by self-interaction order. Within this formalism, we derive convolutional and equivariant networks as symmetry-constrained first-order product interactions, recover Mamba from state-space models as an increase in self-interaction order from one to three, and establish attention as a third-order interaction from composing the same operator. Equivariance is incorporated by constraining the algebraic product via symmetry: translation equivariance yields convolution, while SO(3)-compatibility produces the Clebsch--Gordan product and tensor-field interactions. This perspective recasts architecture design as an algebraic construction problem: the choice of algebra defines admissible interactions, symmetry constrains their structure, and composition generates higher-order architectures. The framework is thus generative. Following these principles, we construct a simplified-algebra dynamical layer that improves both accuracy and throughput over Mamba on CIFAR-10. Experiments show that interaction order affects accuracy non-monotonically and symmetry constraints boost learnability. Product Interactions provides a mathematical methodology for analyzing existing architectures and systematically designing new ones.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。