arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2406.10255cs.CLcs.SI
收藏

WarCov——来自社交平台的大型多标签多模态数据集

WarCov -- Large multilabel and multimodal dataset from social platform

Wroclaw University of Science and Technology(弗罗茨瓦夫理工大学)

Weronika Borek-Marciniec, Pawel Zyblewski, Jakub Klikowski, Pawel Ksieniewicz

AI总结:

本文构建了包含318万余条波兰语社交媒体帖子的WarCov多标签多模态数据集,覆盖疫情与乌克兰战争主题,并展示了从数据获取到模式识别实验的完整整理流程。

摘要:

在分类任务中,从原始数据获取到整理出适合用于评估机器学习模型的数据集,需要一系列步骤,这些步骤往往伴随着高昂的成本。在自然语言处理领域,初步清洗和转换可以自动完成,但获取标签仍然需要人类专家进行合理化的输入。因此,尽管许多文章经常声称“世界充满了数据”,数据科学家却饱受数据短缺之苦。这对于自然语言应用尤为关键,因为该领域不断演进,必须适应新的概念或事件。例如,COVID-19大流行这一主题及其相关词汇在2019年之前大多是难以识别的。出于这一原因,创建新的数据集(包括英语以外的语言)仍然至关重要。本文呈现了一个包含3,187,105条波兰语帖子的集合,这些帖子涉及疫情和乌克兰战争,于2022年发布在流行的社交媒体平台上。该集合不仅包含预处理后的文本,还包含图像,因此也可用于多模态识别任务。标签定义了帖子的主题,并利用伴随帖子的标签(hashtags)创建。本文展示了从数据获取到样本模式识别实验的完整数据集整理过程。

英文摘要:

In the classification tasks, from raw data acquisition to the curation of a dataset suitable for use in evaluating machine learning models, a series of steps - often associated with high costs - are necessary. In the case of Natural Language Processing, initial cleaning and conversion can be performed automatically, but obtaining labels still requires the rationalized input of human experts. As a result, even though many articles often state that "the world is filled with data", data scientists suffer from its shortage. It is crucial in the case of natural language applications, which is constantly evolving and must adapt to new concepts or events. For example, the topic of the COVID-19 pandemic and the vocabulary related to it would have been mostly unrecognizable before 2019. For this reason, creating new datasets, also in languages other than English, is still essential. This work presents a collection of 3~187~105 posts in Polish about the pandemic and the war in Ukraine published on popular social media platforms in 2022. The collection includes not only preprocessed texts but also images so it can be used also for multimodal recognition tasks. The labels define posts' topics and were created using hashtags accompanying the posts. The work presents the process of curating a dataset from acquisition to sample pattern recognition experiments.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑