COOL，一种上下文展望器，及其在问答和其他自然语言处理任务中的应用
COOL, a Context Outlooker, and its Application to Question Answering and other Natural Language Processing Tasks
AI总结：
本文提出COOL，一种用于自然语言处理的展望注意力机制，通过编码局部句法上下文提升变换器模型性能，在问答等任务上取得竞争性结果。
摘要：
视觉展望器通过添加展望注意力（一种局部注意力形式）来实现自注意力机制，从而提升了视觉变换器的性能。在自然语言处理领域，与计算机视觉及其他领域的情况相同，基于变换器的模型构成了大多数处理任务的最先进技术。在这一领域，许多作者也论证并展示了局部上下文的重要性。我们提出了一种用于自然语言处理的展望注意力机制，即COOL。COOL被添加在基于变换器的模型的自注意力层之上，编码局部句法上下文，考虑词邻近性以及比现有方法使用的动态卷积更多的成对约束。对COOL实现与不同基于变换器的模型的比较性经验性能评估证实，在包括问答在内的各种自然语言处理任务中，相对于仅使用原始模型的基线，存在改进的机会。所提出的方法在某些任务上达到了与现有最先进方法竞争的性能。
英文摘要：
Vision outlooker improves the performance of vision transformers, which implements a self-attention mechanism by adding an outlook attention, a form of local attention. In natural language processing, as has been the case in computer vision and other domains, transformer-based models constitute the state-of-the-art for most processing tasks. In this domain, too, many authors have argued and demonstrated the importance of local context. We present an outlook attention mechanism, COOL, for natural language processing. COOL, added on top of the self-attention layers of a transformer-based model, encodes local syntactic context considering word proximity and more pair-wise constraints than dynamic convolution used by existing approaches. A comparative empirical performance evaluation of an implementation of COOL with different transformer-based models confirms the opportunity for improvement over a baseline using the original models alone for various natural language processing tasks, including question answering. The proposed approach achieves competitive performance with existing state-of-the-art methods on some tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
