arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2107.06872cs.NEcs.AIcs.LG
收藏

神经网络中的泛化不需要特征重叠

Generalisation in Neural Networks Does not Require Feature Overlap

Edinburgh Napier University(爱丁堡纳皮尔大学)

Jeff Mitchell, Jeffrey S. Bowers

摘要:

训练和测试数据之间的共享特征需要在人工神经网络中推广,这一直是这些模型的支持者和批评者的共同假设。在这里,我们展示了卷积架构通过将它们应用于两个众所周知的挑战来避免这种限制,基于学习身份函数和学习控制单词序列的规则。在每种情况下,测试集上的成功性能都需要推广到训练数据中不存在的特征,这对于标准连接主义模型通常是不可行的。然而,我们的实验表明,当神经网络结合卷积架构所采用的权重共享时,它们可以成功解决这些问题。在图像处理领域中,这样的架构旨在反映这样的图像所描绘的自然世界的空间平移下的对称性。我们讨论了对称性在这两项任务中的作用及其与概括的联系。

英文摘要:

That shared features between train and test data are required for generalisation in artificial neural networks has been a common assumption of both proponents and critics of these models. Here, we show that convolutional architectures avoid this limitation by applying them to two well known challenges, based on learning the identity function and learning rules governing sequences of words. In each case, successful performance on the test set requires generalising to features that were not present in the training data, which is typically not feasible for standard connectionist models. However, our experiments demonstrate that neural networks can succeed on such problems when they incorporate the weight sharing employed by convolutional architectures. In the image processing domain, such architectures are intended to reflect the symmetry under spatial translations of the natural world that such images depict. We discuss the role of symmetry in the two tasks and its connection to generalisation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑