arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1910.09351cs.LGstat.ML
收藏

复合神经网络的理论研究

Theoretical Investigation of Composite Neural Network

Institute of Information Science, Academia Sinica(中央研究院资讯科学研究所)

Ming-Chuan Yang, Meng Chang Chen

摘要:

本文从理论上研究了复合神经网络的性能。复合神经网络是组合一组预训练和非实例化的神经网络模型的有根有向非循环图,其中预训练的神经网络模型针对特定任务精心制作,并且目标是近似具有实例化权重的特定函数。在复合神经网络中采用这种预训练模型的优点有两个方面。一个是受益于他人的智慧和勤奋,另一个是节省数据准备的努力和培训的资源和时间。但复合神经网络的整体性能仍不清楚。在这项工作中,我们证明了一个复合神经网络,具有很高的概率,比任何预先训练的组件在一定的假设下表现得更好。此外,如果将额外的预训练组件添加到复合网络中,则很有可能提高整体性能。在实证评估中,明显不同的应用支持上述发现。

英文摘要:

This work theoretically investigates the performance of a composite neural network. A composite neural network is a rooted directed acyclic graph combining a set of pre-trained and non-instantiated neural network models, where a pre-trained neural network model is well-crafted for a specific task and targeted to approximate a specific function with instantiated weights. The advantages of adopting such a pre-trained model in a composite neural network are two folds. One is to benefit from other's intelligence and diligence, and the other is saving the efforts in data preparation and resources and time in training. However, the overall performance of composite neural network is still not clear. In this work, we prove that a composite neural network, with high probability, performs better than any of its pre-trained components under certain assumptions. In addition, if an extra pre-trained component is added to a composite network, with high probability the overall performance will be improved. In the empirical evaluations, distinctively different applications support the above findings.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑