arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1606.00540cs.NEcs.LG
收藏

多预训练深度神经网络

Multi-pretrained Deep Neural Network

Science and Technology on Information Systems Engineering Laboratory, Nanjing Research Institute of Electronic and Engineering(信息系统工程重点实验室,南京电子工程研究所);CETC Research Institute of Smart City(中国电子科技集团公司智慧城市研究院)

Zhen Hu, Zhuyin Xue, Tong Cui, Shiqiang Zong, Chenglong He

摘要:

预训练在深度神经网络中有着广泛的应用,其中最著名的预训练模型之一就是深度信念网络(DBN)。不同的预训练模型在预训练过程中所采用的优化公式是不同的。在本文中,我们通过不同的预训练模型对深度神经网络进行了预训练,从而研究了DBN和Stacked Denoising Autoencoder(SDA)在用作预训练模型时的差异。实验结果表明,DBN得到了较好的初始模型。然而,该模型收敛到一个相对较差的模型后,微调过程。然而,经过SDA的第二次预训练,如果微调,模型收敛到一个更好的模型。

英文摘要:

Pretraining is widely used in deep neutral network and one of the most famous pretraining models is Deep Belief Network (DBN). The optimization formulas are different during the pretraining process for different pretraining models. In this paper, we pretrained deep neutral network by different pretraining models and hence investigated the difference between DBN and Stacked Denoising Autoencoder (SDA) when used as pretraining model. The experimental results show that DBN get a better initial model. However the model converges to a relatively worse model after the finetuning process. Yet after pretrained by SDA for the second time the model converges to a better model if finetuned.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑