arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2107.00797cs.LG
收藏

通过连接输入缓解深度双下降

Mitigating deep double descent by concatenating inputs

Rice University(莱斯大学)

John Chen, Qihan Wang, Anastasios Kyrillidis

摘要:

双下降曲线是深度神经网络最有趣的特性之一。它将经典的偏差-方差曲线与现代神经网络的行为进行了对比,发生在样本数量接近参数数量的地方。在这项工作中,我们探讨了双下降现象和深度神经网络设置中的样本数量之间的联系。特别是,我们提出了一个建设,增加现有的数据集,人为地增加样本的数量。该构造经验性地减轻了该设置中的双下降曲线。我们复制现有的工作深双下降,并观察到一个光滑的下降到超参数化的区域,为我们的建设。这既与模型大小有关,也与时期数有关。

英文摘要:

The double descent curve is one of the most intriguing properties of deep neural networks. It contrasts the classical bias-variance curve with the behavior of modern neural networks, occurring where the number of samples nears the number of parameters. In this work, we explore the connection between the double descent phenomena and the number of samples in the deep neural network setting. In particular, we propose a construction which augments the existing dataset by artificially increasing the number of samples. This construction empirically mitigates the double descent curve in this setting. We reproduce existing work on deep double descent, and observe a smooth descent into the overparameterized region for our construction. This occurs both with respect to the model size, and with respect to the number epochs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑