Incremental processing allows interactive systems to respond based on partial inputs, which is a desirable property e.g. in dialogue agents. The currently popular Transformer architecture inherently processes sequences as a whole, abstracting away the notion of time. Recent work attempts to apply Transformers incrementally via restart-incrementality by repeatedly feeding, to an unchanged model, increasingly longer input prefixes to produce partial outputs. However, this approach is computationally costly and does not scale efficiently for long sequences. In parallel, we witness efforts to make Transformers more efficient, e.g. the Linear Transformer (LT) with a recurrence mechanism. In this work, we examine the feasibility of LT for incremental NLU in English. Our results show that the recurrent LT model has better incremental performance and faster inference speed compared to the standard Transformer and LT with restart-incrementality, at the cost of part of the non-incremental (full sequence) quality. We show that the performance drop can be mitigated by training the model to wait for right context before committing to an output and that training with input prefixes is beneficial for delivering correct partial outputs.
When Only Time Will Tell: Interpreting How Transformers Process Local Ambiguities Through the Lens of Restart-Incrementality
University of Potsdam(波茨坦大学);Fraunhofer Heinrich Hertz Institute(弗劳恩霍夫海因里希·赫兹研究所);German Research Center for Artificial Intelligence (DFKI)(德国人工智能研究中心(DFKI))
Incremental Transformer with Deliberation Decoder for Document Grounded Conversations
Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);Northeastern University(东北大学);Huazhong University of Science and Technology(华中科技大学);Tencent Inc(腾讯公司)
Generalization in NLI: Ways (Not) To Go Beyond Simple Heuristics
The University of Texas at Dallas(德克萨斯大学达拉斯分校);RIKEN Center for Computational Science(理化学研究所计算科学中心);Tokyo Institute of Technology(东京工业大学);Center for Social Data Science, University of Copenhagen(哥本哈根大学社会数据科学中心)
Future-Guided Incremental Transformer for Simultaneous Translation
Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学);Huawei Noah’s Ark Lab(华为诺亚方舟实验室)