arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2204.13154cs.LG
收藏

神经网络中的注意力机制:它从哪里来,到哪里去

Attention Mechanism in Neural Networks: Where it Comes and Where it Goes

University of Leuven (KU Leuven)(鲁汶大学)

Derya Soydaner

摘要:

很久以前,在机器学习文献中,引入了将受人类视觉系统启发的机制并入神经网络的想法。这种思想被命名为注意力机制,它经历了一个漫长的发展时期。今天,许多作品都致力于这一想法在各种任务。最近表现出了显著的业绩。本文的目的是提供一个概述,从早期的工作,寻找方法来实现注意的想法与神经网络,直到最近的趋势。这次审查强调了在这一进展过程中有关不同任务的重要里程碑。通过这种方式,本研究旨在为研究人员提供一个路线图,以探索目前的发展和获得灵感的新方法以外的关注。

英文摘要:

A long time ago in the machine learning literature, the idea of incorporating a mechanism inspired by the human visual system into neural networks was introduced. This idea is named the attention mechanism, and it has gone through a long development period. Today, many works have been devoted to this idea in a variety of tasks. Remarkable performance has recently been demonstrated. The goal of this paper is to provide an overview from the early work on searching for ways to implement attention idea with neural networks until the recent trends. This review emphasizes the important milestones during this progress regarding different tasks. By this way, this study aims to provide a road map for researchers to explore the current development and get inspired for novel approaches beyond the attention.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑