arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2304.07258cs.CLcs.AIcs.LG
收藏

先学习可能性,再选择最优解:通过基于文本的游戏解开语言中的一对多关系

Learn What Is Possible, Then Choose What Is Best: Disentangling One-To-Many Relations in Language Through Text-based Games

University of Nottingham(诺丁汉大学);Nokia Bell Labs(诺基亚贝尔实验室)

Benjamin Towle, Ke Zhou

AI总结:

针对NLP预训练数据一对多结构下现有一对一训练方法的局限,提出PASA方法,通过离散潜变量捕捉行为范围并经知识蒸馏迁移至学生模型,在Jericho Walkthroughs数据集上较SOTA最高提升49%。

摘要:

在大规模自监督语料上预训练、再进行任务特定微调的语言模型已成为自然语言处理(NLP)的主流范式。这些预训练数据集通常具有一对多结构——例如在对话中,给定上下文存在许多有效回复。然而,其中只有部分回复是下游任务中期望的。这就提出了一个问题:我们应如何训练模型,使其能模拟期望行为,而非非期望行为。现有方法在一对一设置下训练——单个对话上下文仅对应一个目标回复——导致模型仅学习预测平均回复,而忽略了所有可能的回复范围。我们以基于文本的游戏为测试平台,提出的PASA方法使用离散潜变量来捕捉更大规模预训练数据集中呈现的不同行为范围。随后我们使用知识蒸馏将后验概率分布蒸馏到学生模型中。该概率分布远比仅从数据集的硬目标学习更丰富,因此能让学生模型从教师模型学到的更丰富的动作范围中受益。实验结果显示,在Jericho Walkthroughs数据集上,该方法相比此前的最优模型实现了最高达49%的实证提升。

英文摘要:

Language models pre-trained on large self-supervised corpora, followed by task-specific fine-tuning has become the dominant paradigm in NLP. These pre-training datasets often have a one-to-many structure--e.g. in dialogue there are many valid responses for a given context. However, only some of these responses will be desirable in our downstream task. This raises the question of how we should train the model such that it can emulate the desirable behaviours, but not the undesirable ones. Current approaches train in a one-to-one setup--only a single target response is given for a single dialogue context--leading to models only learning to predict the average response, while ignoring the full range of possible responses. Using text-based games as a testbed, our approach, PASA, uses discrete latent variables to capture the range of different behaviours represented in our larger pre-training dataset. We then use knowledge distillation to distil the posterior probability distribution into a student model. This probability distribution is far richer than learning from only the hard targets of the dataset, and thus allows the student model to benefit from the richer range of actions the teacher model has learned. Results show up to 49% empirical improvement over the previous state-of-the-art model on the Jericho Walkthroughs dataset.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑