记忆增强生成对抗 Transformer
Memory-Augmented Generative Adversarial Transformers
AI总结：
本文提出记忆增强的生成对抗 Transformer，通过外部记忆库与额外注意力层融入事实等信息，以支持事实问答和风格适应。
摘要：
依赖大型语言模型（如 Transformer）的对话式 AI 系统，难以将外部数据（如事实）与其生成的语言交织在一起。原始 Transformer 架构并非为高精度回答事实性问题而设计。本文研究了解决该问题的一种可能途径。我们提议扩展标准 Transformer 架构，增加一个保存额外信息（例如从知识库中提取的事实）的记忆库，以及一个用于访问该记忆的额外注意力层。我们将这种增强记忆加入一种受生成对抗网络启发的 Transformer 架构中。该设置允许对 Transformer 生成的语言实施任意的适切性条件。我们首先展示如何部署这一机制来处理目标导向对话中的事实性问题。其次，我们证明我们的方法也可用于诸如风格适应之类的应用：即根据某些风格性（外部）约束调整话语，例如对话中人类交谈者的社会属性。
英文摘要：
Conversational AI systems that rely on Large Language Models, like Transformers, have difficulty interweaving external data (like facts) with the language they generate. Vanilla Transformer architectures are not designed for answering factual questions with high accuracy. This paper investigates a possible route for addressing this problem. We propose to extend the standard Transformer architecture with an additional memory bank holding extra information (such as facts drawn from a knowledge base), and an extra attention layer for addressing this memory. We add this augmented memory to a Generative Adversarial Network-inspired Transformer architecture. This setup allows for implementing arbitrary felicity conditions on the generated language of the Transformer. We first demonstrate how this machinery can be deployed for handling factual questions in goal-oriented dialogues. Secondly, we demonstrate that our approach can be useful for applications like {\it style adaptation} as well: the adaptation of utterances according to certain stylistic (external) constraints, like social properties of human interlocutors in dialogues.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
