arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1912.05877cs.CLcs.AI
收藏

机器语言模型向人级语言理解的扩展

Extending Machine Language Models toward Human-Level Language Understanding

Stanford University(斯坦福大学);DeepMind;Bosch Center for Artificial Intelligence(博世人工智能中心);Google Research(谷歌研究院);LMU Munich(慕尼黑大学)

James L. McClelland, Felix Hill, Maja Rudolph, Jason Baldridge, Hinrich Schütze

摘要:

语言对人类智力至关重要,但它的作用究竟是什么?我们认为语言是理解和交流环境的系统的一部分。人类理解和交流情况的能力逐渐从经验中产生,并依赖于生物神经网络的领域一般原则:基于连接的学习,分布式表示,以及上下文敏感,基于相互约束满足的处理。当前的人工语言处理系统依赖于体现在人工神经网络中的相同领域的一般原理。事实上,这一领域的最新进展依赖于\emph{query-based attention},它扩展了这些系统利用上下文的能力,并促成了显着的突破。然而,目前大多数模型只关注语言内部任务,限制了它们执行依赖于理解情况的任务的能力。这些系统也缺乏对固定上下文范围之外的先前情况的内容的记忆。我们描述了大脑的分布式理解系统的组织,其中包括一个快速学习系统,解决记忆问题。我们勾画了一个框架,为未来的理解模型,同样借鉴认知神经科学和人工智能,并利用查询为基础的注意。我们强调相关的当前方向,并考虑在计算系统中完全捕获人类水平的语言理解所需的进一步发展。

英文摘要:

Language is crucial for human intelligence, but what exactly is its role? We take language to be a part of a system for understanding and communicating about situations. The human ability to understand and communicate about situations emerges gradually from experience and depends on domain-general principles of biological neural networks: connection-based learning, distributed representation, and context-sensitive, mutual constraint satisfaction-based processing. Current artificial language processing systems rely on the same domain general principles, embodied in artificial neural networks. Indeed, recent progress in this field depends on \emph{query-based attention}, which extends the ability of these systems to exploit context and has contributed to remarkable breakthroughs. Nevertheless, most current models focus exclusively on language-internal tasks, limiting their ability to perform tasks that depend on understanding situations. These systems also lack memory for the contents of prior situations outside of a fixed contextual span. We describe the organization of the brain's distributed understanding system, which includes a fast learning system that addresses the memory problem. We sketch a framework for future models of understanding drawing equally on cognitive neuroscience and artificial intelligence and exploiting query-based attention. We highlight relevant current directions and consider further developments needed to fully capture human-level language understanding in a computational system.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑