评估序列间模型中的增量
Assessing incrementality in sequence-to-sequence models
摘要：
编码器-解码器模型自问世以来，已成功地应用于计算语言学中的一系列问题。最近的成功主要是由于使用不同的注意力机制的变化，但他们的认知能力是值得怀疑的。特别是，因为过去的表示可以在任何时间点重新访问，以注意力为中心的方法似乎缺乏激励，以建立增量更多的信息表示传入的句子。这种处理方式与人们认为的人类处理语言的方式形成鲜明对比：不断快速地整合遇到的新信息。在这项工作中，我们提出了三个新的指标来评估有和没有注意力机制的RNN的行为，并确定不同模型类型处理句子的方式的关键差异。
英文摘要：
Since their inception, encoder-decoder models have successfully been applied to a wide array of problems in computational linguistics. The most recent successes are predominantly due to the use of different variations of attention mechanisms, but their cognitive plausibility is questionable. In particular, because past representations can be revisited at any point in time, attention-centric methods seem to lack an incentive to build up incrementally more informative representations of incoming sentences. This way of processing stands in stark contrast with the way in which humans are believed to process language: continuously and rapidly integrating new information as it is encountered. In this work, we propose three novel metrics to assess the behavior of RNNs with and without an attention mechanism and identify key differences in the way the different model types process sentences.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
