TextBox 2.0:一个基于预训练语言模型的文本生成库
TextBox 2.0: A Text Generation Library with Pre-trained Language Models
摘要:
为了促进文本生成的研究,本文提出了一个全面和统一的库,TextBox 2.0,专注于使用预先训练的语言模型(PLM)。为了全面,我们的库涵盖了$13$常见的文本生成任务及其相应的$83$数据集,并进一步整合了$45$ PLM,涵盖了通用、翻译、中文、对话、可控、提炼、提示和轻量级PLM。我们还实施了$4$高效的培训策略,并提供了$4$生成目标,用于从头开始预培训新的PLM。为了统一,我们设计了支持整个研究流程(从数据加载到训练和评估)的接口,确保每个步骤都可以以统一的方式完成。尽管功能丰富,但通过友好的Python API或命令行,很容易使用我们的库。为了验证我们的图书馆的有效性,我们进行了广泛的实验和四种类型的研究方案。该项目发布在链接:https://github.com/RUCAIBox/TextBox。
英文摘要:
To facilitate research on text generation, this paper presents a comprehensive and unified library, TextBox 2.0, focusing on the use of pre-trained language models (PLMs). To be comprehensive, our library covers $13$ common text generation tasks and their corresponding $83$ datasets and further incorporates $45$ PLMs covering general, translation, Chinese, dialogue, controllable, distilled, prompting, and lightweight PLMs. We also implement $4$ efficient training strategies and provide $4$ generation objectives for pre-training new PLMs from scratch. To be unified, we design the interfaces to support the entire research pipeline (from data loading to training and evaluation), ensuring that each step can be fulfilled in a unified way. Despite the rich functionality, it is easy to use our library, either through the friendly Python API or command line. To validate the effectiveness of our library, we conduct extensive experiments and exemplify four types of research scenarios. The project is released at the link: https://github.com/RUCAIBox/TextBox.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。