arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2005.06420cs.CLcs.LG
收藏

计算语言学在深度学习中的崛起

The Unstoppable Rise of Computational Linguistics in Deep Learning

Idiap Research Institute(伊迪亚普研究院)

James Henderson

摘要:

在本文中,我们追溯了神经网络应用于自然语言理解任务的历史,并确定了语言的本质对神经网络体系结构的发展做出的关键贡献。我们重点讨论了变量绑定及其实例化在基于注意力模型中的重要性,并指出Transformer不是一个序列模型,而是一个诱导结构模型。这种观点导致预测自然语言理解的深度学习架构研究所面临的挑战。

英文摘要:

In this paper, we trace the history of neural networks applied to natural language understanding tasks, and identify key contributions which the nature of language has made to the development of neural network architectures. We focus on the importance of variable binding and its instantiation in attention-based models, and argue that Transformer is not a sequence model but an induced-structure model. This perspective leads to predictions of the challenges facing research in deep learning architectures for natural language understanding.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑