TorontoCL参加CMCL 2021共享任务：RoBERTa与多阶段微调用于眼动追踪预测
TorontoCL at CMCL 2021 Shared Task: RoBERTa with Multi-Stage Fine-Tuning for Eye-Tracking Prediction
摘要：
阅读过程中的眼动数据是理解语言理解过程的有用信息来源。在本文中，我们描述了我们提交给CMCL 2021预测人类阅读模式的共享任务。我们的模型使用带有回归层的RoBERTa来预测5个眼动追踪特征。我们分两个阶段对模型进行训练：首先在Provo语料库（另一个眼动追踪数据集）上进行微调，然后在任务数据上进行微调。我们比较了不同的Transformer模型，并应用集成方法来提高性能。我们最终提交的作品获得了3.929的MAE分数，在参与这项共同任务的13支团队中排名第三。
英文摘要：
Eye movement data during reading is a useful source of information for understanding language comprehension processes. In this paper, we describe our submission to the CMCL 2021 shared task on predicting human reading patterns. Our model uses RoBERTa with a regression layer to predict 5 eye-tracking features. We train the model in two stages: we first fine-tune on the Provo corpus (another eye-tracking dataset), then fine-tune on the task data. We compare different Transformer models and apply ensembling methods to improve the performance. Our final submission achieves a MAE score of 3.929, ranking 3rd place out of 13 teams that participated in this shared task.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
