arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1909.09595cs.CLcs.LGcs.NE
收藏

SANVis:理解自我注意网络的可视化分析

SANVis: Visual Analytics for Understanding Self-Attention Networks

Korea University(高丽大学);University of Maryland(马里兰大学);IBM Research(IBM研究院);University of Waterloo(滑铁卢大学);FXPAL;NCSOFT Co. LTD.(NCSOFT有限公司)

Cheonbok Park, Inyoup Na, Yongjang Jo, Sungbok Shin, Jaehyo Yoo, Bum Chul Kwon, Jian Zhao, Hyungjong Noh, Yeonsoo Lee, Jaegul Choo

摘要:

注意力网络是一种受人类注意力机制启发的深度神经网络架构,在图像描述、机器翻译和许多其他应用中取得了重大成功。最近,它们已经进一步发展成一种称为多头自注意网络的高级方法,该方法可以对一组输入向量进行编码,例如,一个句子中的词向量,转化成另一组向量。这种编码的目的是在一个集合中同时捕捉不同的句法和语义特征,每个特征对应一个特定的注意力中心,从而形成多头注意力。同时,增加的模型复杂性使用户无法轻松理解和操纵模型的内部工作。为了应对这些挑战,我们提出了一个名为SANVis的可视化分析系统,它可以帮助用户了解多头自注意网络的行为和特征。使用一个最先进的自我注意力模型称为Transformer,我们演示了SANVis在机器翻译任务中的使用场景。我们的系统可在http://short.sanvis.org上获得

英文摘要:

Attention networks, a deep neural network architecture inspired by humans' attention mechanism, have seen significant success in image captioning, machine translation, and many other applications. Recently, they have been further evolved into an advanced approach called multi-head self-attention networks, which can encode a set of input vectors, e.g., word vectors in a sentence, into another set of vectors. Such encoding aims at simultaneously capturing diverse syntactic and semantic features within a set, each of which corresponds to a particular attention head, forming altogether multi-head attention. Meanwhile, the increased model complexity prevents users from easily understanding and manipulating the inner workings of models. To tackle the challenges, we present a visual analytics system called SANVis, which helps users understand the behaviors and the characteristics of multi-head self-attention networks. Using a state-of-the-art self-attention model called Transformer, we demonstrate usage scenarios of SANVis in machine translation tasks. Our system is available at http://short.sanvis.org

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑