模块化作为神经网络学习中复杂性管理的一种手段
Modularity as a Means for Complexity Management in Neural Networks Learning
摘要:
训练具有大量参数或复杂架构的神经网络(NN)会产生使优化过程复杂化的不良现象。为了解决这个问题,我们提出了第一个模块化的NN设计方法,其中NN被分解为一个控制模块和几个功能模块,实现原始操作。我们通过比较单片和模块化NN在列表排序问题上的性能来说明模块化概念,并在训练速度,训练稳定性和可维护性方面显示出优势。我们还讨论了模块化NN中出现的一些问题。
英文摘要:
Training a Neural Network (NN) with lots of parameters or intricate architectures creates undesired phenomena that complicate the optimization process. To address this issue we propose a first modular approach to NN design, wherein the NN is decomposed into a control module and several functional modules, implementing primitive operations. We illustrate the modular concept by comparing performances between a monolithic and a modular NN on a list sorting problem and show the benefits in terms of training speed, training stability and maintainability. We also discuss some questions that arise in modular NNs.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。