探究语言模型中基于掩码的数据生成
Investigating Masking-based Data Generation in Language Models
AI总结：
本文探究了基于掩码语言模型（MLM）的数据增强方法在自然语言处理下游任务中的更广泛应用，指出该技术能简单高效地提升模型性能。
摘要：
自BERT问世以来，预训练语言模型的显著地位定义了当前的自然语言处理（NLP）时代。BERT及类似架构模型的一个特征是掩码语言建模目标，即故意掩码输入的一部分，并训练模型预测这部分被掩码的信息。数据增强是一种在机器学习中广泛使用的数据驱动技术，包括计算机视觉和自然语言处理等领域，通过指定技术人为扩充训练数据集以提升模型性能。掩码语言模型（MLM）作为BERT的核心训练特征，为在NLP任务的Transformer模型上进行有效预训练引入了新方法。近期研究利用MLM为NLP下游任务生成人工增强数据。实验结果表明，基于掩码的数据增强方法提供了一种简单但有效的途径来提升模型性能。本文探索并讨论了基于MLM的这些数据增强方法的更广泛应用。
英文摘要：
The current era of natural language processing (NLP) has been defined by the prominence of pre-trained language models since the advent of BERT. A feature of BERT and models with similar architecture is the objective of masked language modeling, in which part of the input is intentionally masked and the model is trained to predict this piece of masked information. Data augmentation is a data-driven technique widely used in machine learning, including research areas like computer vision and natural language processing, to improve model performance by artificially augmenting the training data set by designated techniques. Masked language models (MLM), an essential training feature of BERT, have introduced a novel approach to perform effective pre-training on Transformer based models in natural language processing tasks. Recent studies have utilized masked language model to generate artificially augmented data for NLP downstream tasks. The experimental results show that Mask based data augmentation method provides a simple but efficient approach to improve the model performance. In this paper, we explore and discuss the broader utilization of these data augmentation methods based on MLM.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
