arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2512.13480cs.LG
收藏

随机矩阵元素级调制用于高效的神经层

Element-wise Modulation of Random Matrices for Efficient Neural Layers

Maksymilian Szorc

AI总结:

通过元素级调制随机矩阵,提出PRP层以高效减少全连接层的计算开销,同时保持模型精度。

摘要:

全连接层是深度神经网络中内存和计算开销的主要来源,由于其密集且通常冗余的参数化。尽管存在各种压缩技术,但它们经常引入复杂的工程权衡或降低模型性能。我们提出了参数化随机投影(PRP)层,这是一种新的方法,通过利用固定随机矩阵并由轻量级、可学习的元素级参数调制,将特征混合与适应解耦。该架构将可训练参数数量大幅减少到线性规模,同时在各种基准测试中保持可靠的准确性。该设计为架构扩展和在资源受限环境中部署提供了稳定且计算高效的解决方案。

英文摘要:

Fully connected layers are a primary source of memory and computational overhead in deep neural networks due to their dense, often redundant parameterization. While various compression techniques exist, they frequently introduce complex engineering trade-offs or degrade model performance. We propose the Parametrized Random Projection (PRP) layer, a novel approach that decouples feature mixing from adaptation by utilizing a fixed random matrix modulated by lightweight, learnable element-wise parameters. This architecture drastically reduces the trainable parameter count to a linear scale while retaining reliable accuracy across various benchmarks. The design serves as a stable, computationally efficient solution for architectural scaling and deployment in resource-limited settings.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑