arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2101.08386cs.LGcs.CL
收藏

不变性、编码和泛化:用神经网络学习同一性效应

Invariance, encodings, and generalization: learning identity effects with neural networks

Concordia University(康考迪亚大学);Simon Fraser University(西蒙菲莎大学)

S. Brugiapaglia, M. Liu, P. Tupper

摘要:

通常在语言和其他认知领域,一个对象的两个组成部分是否相同决定了它是否是良好的形式。我们称这种约束为身份效应。当开发一个系统来从示例中学习格式良好性时,很容易建立识别效果。但是,在没有明确指导的情况下,是否可以从数据中学习身份效应?我们提供了一个框架,在这个框架中,我们可以严格地证明,满足简单标准的算法不能做出正确的推断。然后,我们证明了一大类学习算法,包括通过基于梯度的算法(如随机梯度下降或亚当方法)训练的深度前馈神经网络,满足我们的标准,这取决于输入的编码。在一些更广泛的情况下,我们能够提供网络必然会错误分类的对抗性示例。最后,我们证明了我们的理论与计算实验中,我们探讨了不同的输入编码算法的能力,以推广到新的输入的效果。这使我们能够显示出类似的效果,理论预测的更现实的方法,违反了我们的理论结果的一些条件。

英文摘要:

Often in language and other areas of cognition, whether two components of an object are identical or not determines if it is well formed. We call such constraints identity effects. When developing a system to learn well-formedness from examples, it is easy enough to build in an identify effect. But can identity effects be learned from the data without explicit guidance? We provide a framework in which we can rigorously prove that algorithms satisfying simple criteria cannot make the correct inference. We then show that a broad class of learning algorithms including deep feedforward neural networks trained via gradient-based algorithms (such as stochastic gradient descent or the Adam method) satisfy our criteria, dependent on the encoding of inputs. In some broader circumstances we are able to provide adversarial examples that the network necessarily classifies incorrectly. Finally, we demonstrate our theory with computational experiments in which we explore the effect of different input encodings on the ability of algorithms to generalize to novel inputs. This allows us to show similar effects to those predicted by theory for more realistic methods that violate some of the conditions of our theoretical results.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑