arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2109.09371cs.AIcs.CLcs.NEstat.ML
收藏

从零开始学习自然语言生成

Learning Natural Language Generation from Scratch

CMAP, École Polytechnique, Institut Polytechnique de Paris(巴黎综合理工学院应用数学中心,巴黎理工学院);Samovar, Télécom SudParis, département CITI, TIPIC, Institut Polytechnique de Paris(萨莫瓦尔实验室,南巴黎电信学院,CITI系,TIPIC实验室,巴黎理工学院);Deep Mind(深度思维);Google Research, Brain Team(谷歌研究院大脑团队)

Alice Martin Donati, Guillaume Quispe, Charles Ollion, Sylvain Le Corff, Florian Strub, Olivier Pietquin

摘要:

本文介绍了TRUNCATED REINFORCENT LEARNING FOR LANGUAGE(TrufLL),这是一种仅使用强化学习(RL)从头开始训练条件语言模型的新颖方法。由于RL方法不能成功地扩展到大的动作空间,我们使用通用的语言模型动态截断词汇空间。因此,TrufLL能够训练语言代理,只与其环境交互,没有任何特定于任务的先验知识;它只与任务无关的语言模型指导。有趣的是,这种方法避免了对标记数据集的依赖,并从本质上减少了预先训练的政策缺陷,如语言或暴露偏差。我们评估TrufLL的两个视觉问题生成任务,我们报告的性能和语言指标,这wethen证实了人类的评价积极的结果。据我们所知,这是第一种(几乎)从零开始成功学习语言生成策略的方法。

英文摘要:

This paper introduces TRUncated ReinForcement Learning for Language (TrufLL), an original ap-proach to train conditional language models from scratch by only using reinforcement learning (RL). AsRL methods unsuccessfully scale to large action spaces, we dynamically truncate the vocabulary spaceusing a generic language model. TrufLL thus enables to train a language agent by solely interacting withits environment without any task-specific prior knowledge; it is only guided with a task-agnostic languagemodel. Interestingly, this approach avoids the dependency to labelled datasets and inherently reduces pre-trained policy flaws such as language or exposure biases. We evaluate TrufLL on two visual questiongeneration tasks, for which we report positive results over performance and language metrics, which wethen corroborate with a human evaluation. To our knowledge, it is the first approach that successfullylearns a language generation policy (almost) from scratch.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑