大规模语言模型领域自适应训练的局限性探讨
Exploring the Limits of Domain-Adaptive Training for Detoxifying Large-Scale Language Models
摘要：
预先训练的语言模型（LM）很容易生成有毒语言。在这项工作中，我们系统地探索领域自适应训练，以减少语言模型的毒性。我们从训练语料、模型大小和参数效率三个维度进行研究。对于训练语料库，我们建议利用LM的生成能力，为领域自适应训练生成无毒数据集，这可以减轻暴露偏差，并且比使用策划的预训练语料库更有效。我们证明了自生成方法在自动和人工评估中始终优于各种模型大小的现有基线，即使它使用1/3较小的训练语料库。然后，我们全面研究了解毒LM，其参数大小从126 M到530 B（比GPT-3大3倍），这是以前从未研究过的规模。我们发现，i）在相同的预训练语料库中，大型LM与较小的LM具有相似的毒性水平，ii）大型LM需要更多的努力来解毒。我们还探索参数高效的戒毒培训方法。我们证明了在LM中添加和训练仅适配器层不仅节省了大量参数，而且比大规模模型的整个模型自适应在毒性和困惑之间实现了更好的权衡。
英文摘要：
Pre-trained language models (LMs) are shown to easily generate toxic language. In this work, we systematically explore domain-adaptive training to reduce the toxicity of language models. We conduct this study on three dimensions: training corpus, model size, and parameter efficiency. For the training corpus, we propose to leverage the generative power of LMs and generate nontoxic datasets for domain-adaptive training, which mitigates the exposure bias and is shown to be more data-efficient than using a curated pre-training corpus. We demonstrate that the self-generation method consistently outperforms the existing baselines across various model sizes on both automatic and human evaluations, even when it uses a 1/3 smaller training corpus. We then comprehensively study detoxifying LMs with parameter sizes ranging from 126M up to 530B (3x larger than GPT-3), a scale that has never been studied before. We find that i) large LMs have similar toxicity levels as smaller ones given the same pre-training corpus, and ii) large LMs require more endeavor to detoxify. We also explore parameter-efficient training methods for detoxification. We demonstrate that adding and training adapter-only layers in LMs not only saves a lot of parameters but also achieves a better trade-off between toxicity and perplexity than whole model adaptation for the large-scale models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
