arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2110.08015cs.CL
收藏

使用序列到序列Transformers的危机领域自适应

Crisis Domain Adaptation Using Sequence-to-sequence Transformers

University College Dublin(都柏林大学学院)

Congcong Wang, Paul Nulty, David Lillis

摘要:

社交媒体上的用户生成内容(UGC)可以作为危机情况下紧急救援人员的关键信息来源。然而,由于所涉及的数量,需要计算技术来有效地过滤和优先考虑这些内容,因为它出现在新出现的事件。在文献中,这些技术是使用以前危机中的注释内容进行训练的。在本文中,我们研究如何先验知识可以最好地利用新的危机,检查在何种程度上类似类型的危机事件更适合适应新的事件(跨领域自适应)。鉴于最近成功的变压器在各种语言处理任务,我们提出CAST:危机领域自适应利用序列到序列Transformers的方法。我们使用两个主要的危机相关的消息分类数据集来评估CAST。我们的实验表明,我们的基于CAST的最佳运行,而不使用任何目标数据实现了最先进的性能在域内和跨域的上下文中。此外,CAST在用较大的语言模型训练时,在一对一跨领域自适应中特别有效。在多对一适应多个危机共同使用的源域,CAST进一步提高了其性能。此外,我们发现,更多的相似事件更有可能带来更好的适应性能,而微调使用不同的事件并没有帮助适应。为了提高可重复性,我们向社区开放了我们的代码。

英文摘要:

User-generated content (UGC) on social media can act as a key source of information for emergency responders in crisis situations. However, due to the volume concerned, computational techniques are needed to effectively filter and prioritise this content as it arises during emerging events. In the literature, these techniques are trained using annotated content from previous crises. In this paper, we investigate how this prior knowledge can be best leveraged for new crises by examining the extent to which crisis events of a similar type are more suitable for adaptation to new events (cross-domain adaptation). Given the recent successes of transformers in various language processing tasks, we propose CAST: an approach for Crisis domain Adaptation leveraging Sequence-to-sequence Transformers. We evaluate CAST using two major crisis-related message classification datasets. Our experiments show that our CAST-based best run without using any target data achieves the state of the art performance in both in-domain and cross-domain contexts. Moreover, CAST is particularly effective in one-to-one cross-domain adaptation when trained with a larger language model. In many-to-one adaptation where multiple crises are jointly used as the source domain, CAST further improves its performance. In addition, we find that more similar events are more likely to bring better adaptation performance whereas fine-tuning using dissimilar events does not help for adaptation. To aid reproducibility, we open source our code to the community.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑