LRG在SemEval-2021的任务4：使用扩充、语言特征和投票提高抽象词的阅读理解
LRG at SemEval-2021 Task 4: Improving Reading Comprehension with Abstract Words using Augmentation, Linguistic Features and Voting
摘要：
在这篇文章中，我们提出了我们的方法SemEval-2021任务-4：阅读理解的抽象意义。给定一个填空式问题和相应的上下文，任务是从5个选项中预测最合适的单词。这个任务中有三个子任务：不可感知性（子任务-I），非特异性（子任务-II）和交叉（子任务-III）。我们使用基于transformers的模型的编码器，这些模型在掩码语言建模（MLM）任务上进行了预训练，以构建我们的填空（FitB）模型。此外，模型的不可感知性，我们定义了某些语言特征，和模型的非特异性，我们利用词汇数据库提供的上位词和下位词的信息。具体来说，对于非特异性，我们尝试增强技术和其他统计技术。我们还提出了变体，即块投票和最大上下文，以照顾输入长度限制BERT等，此外，我们进行了彻底的消融研究，并使用集成的解释我们的预测在几个样本。我们最好的提交达到75.31%和77.84%的准确率，在测试集的subtask-I和subtask-II，分别。对于子任务III，我们实现了65.64%和62.27%的准确率。
英文摘要：
In this article, we present our methodologies for SemEval-2021 Task-4: Reading Comprehension of Abstract Meaning. Given a fill-in-the-blank-type question and a corresponding context, the task is to predict the most suitable word from a list of 5 options. There are three sub-tasks within this task: Imperceptibility (subtask-I), Non-Specificity (subtask-II), and Intersection (subtask-III). We use encoders of transformers-based models pre-trained on the masked language modelling (MLM) task to build our Fill-in-the-blank (FitB) models. Moreover, to model imperceptibility, we define certain linguistic features, and to model non-specificity, we leverage information from hypernyms and hyponyms provided by a lexical database. Specifically, for non-specificity, we try out augmentation techniques, and other statistical techniques. We also propose variants, namely Chunk Voting and Max Context, to take care of input length restrictions for BERT, etc. Additionally, we perform a thorough ablation study, and use Integrated Gradients to explain our predictions on a few samples. Our best submissions achieve accuracies of 75.31% and 77.84%, on the test sets for subtask-I and subtask-II, respectively. For subtask-III, we achieve accuracies of 65.64% and 62.27%.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
