ERNIE在SemEval-2020任务10：通过预训练的语言模型学习单词强调选择
ERNIE at SemEval-2020 Task 10: Learning Word Emphasis Selection by Pre-trained Language Model
摘要：
本文介绍了ERNIE团队设计的系统，该系统在SemEval-2020任务10：视觉媒体中书面文本的重点选择中获得了第一名。给定一个句子，我们被要求找出最重要的单词作为自动设计的建议。我们利用无监督的预训练模型，并根据我们的任务微调这些模型。经过我们的调查，我们发现以下模型在这个任务中取得了优异的性能：ERNIE 2.0，XLM-ROBERTA，ROBERTA和ALBERT。我们结合了逐点回归损失和成对排名损失，这更接近最终的M atchm度量来微调我们的模型。我们还发现，额外的特征工程和数据增强可以帮助提高性能。我们的最佳模型获得了0.823的最高分，在各种指标中排名第一
英文摘要：
This paper describes the system designed by ERNIE Team which achieved the first place in SemEval-2020 Task 10: Emphasis Selection For Written Text in Visual Media. Given a sentence, we are asked to find out the most important words as the suggestion for automated design. We leverage the unsupervised pre-training model and finetune these models on our task. After our investigation, we found that the following models achieved an excellent performance in this task: ERNIE 2.0, XLM-ROBERTA, ROBERTA and ALBERT. We combine a pointwise regression loss and a pairwise ranking loss which is more close to the final M atchm metric to finetune our models. And we also find that additional feature engineering and data augmentation can help improve the performance. Our best model achieves the highest score of 0.823 and ranks first for all kinds of metrics
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
