UmBERTo-MTSA @ AcCompl-It:通过自监督注释的多任务学习来改善复杂性和可接受性预测
UmBERTo-MTSA @ AcCompl-It: Improving Complexity and Acceptability Prediction with Multi-task Learning on Self-Supervised Annotations
摘要:
本文描述了一种自监督的数据增强方法,用于在只有适量标记数据可用时提高学习模型的性能。原始模型的多个副本最初在下游任务上进行训练。然后,他们的预测用于注释大量未标记的示例。最后,对所得到的训练集的并行注释执行多任务训练,并通过对注释者特定的头部预测进行平均来获得最终分数。神经语言模型在Evalita 2020的AcCompl-it共享任务的背景下使用该程序进行微调,从而在预测质量方面获得了相当大的改进。
英文摘要:
This work describes a self-supervised data augmentation approach used to improve learning models' performances when only a moderate amount of labeled data is available. Multiple copies of the original model are initially trained on the downstream task. Their predictions are then used to annotate a large set of unlabeled examples. Finally, multi-task training is performed on the parallel annotations of the resulting training set, and final scores are obtained by averaging annotator-specific head predictions. Neural language models are fine-tuned using this procedure in the context of the AcCompl-it shared task at EVALITA 2020, obtaining considerable improvements in prediction quality.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。