arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2312.08968cs.CLcs.AI
收藏

在俄罗斯社交媒体中检测价值表达文本帖子

Detecting value-expressive text posts in Russian social media

University of Waterloo(滑铁卢大学);Institute of Sociology FCTAS RAS(俄罗斯科学院联邦社会学研究中心社会学研究所)

Maria Milkova, Maksim Rudnev, Lidia Okolskaya

AI总结:

本文提出了一种在俄罗斯社交媒体VKontakte中准确检测价值表达文本帖子的模型,利用人类和AI辅助注释及主动学习方法,最终通过微调rubert-tiny2模型实现了高检测精度。

摘要:

基本价值观是与理想终点状态相关的概念或信念,超越具体情境。研究社交媒体中的个人价值观可以揭示社会价值观如何演变,尤其是在基于刺激的方法(如调查)效率低下时,例如在难以接触的人群中。另一方面,用户生成的内容是由大量使用刻板、文化定义的言语构造驱动的,而非个人价值观的真实表达。我们旨在找到一个能够准确检测俄罗斯社交媒体VKontakte上价值表达帖子的模型。训练数据集包含5,035条帖子,由三位专家、304名众包人员和ChatGPT进行标注。众包人员和专家在分类帖子时仅表现出中等程度的一致性。ChatGPT更加一致,但难以检测垃圾信息。我们应用了结合人类和AI辅助注释的集成方法,涉及主动学习方法,随后使用各种预训练Transformer语言模型的嵌入进行训练。最佳性能由微调的rubert-tiny2模型实现,产生高质量的价值检测(F1=0.77,F1-macro=0.83)。该模型为研究俄罗斯社交媒体用户之间的价值观提供了关键一步。

英文摘要:

Basic values are concepts or beliefs which pertain to desirable end-states and transcend specific situations. Studying personal values in social media can illuminate how and why societal values evolve especially when the stimuli-based methods, such as surveys, are inefficient, for instance, in hard-to-reach populations. On the other hand, user-generated content is driven by the massive use of stereotyped, culturally defined speech constructions rather than authentic expressions of personal values. We aimed to find a model that can accurately detect value-expressive posts in Russian social media VKontakte. A training dataset of 5,035 posts was annotated by three experts, 304 crowd-workers and ChatGPT. Crowd-workers and experts showed only moderate agreement in categorizing posts. ChatGPT was more consistent but struggled with spam detection. We applied an ensemble of human- and AI-assisted annotation involving active learning approach, subsequently trained several classification models using embeddings from various pre-trained transformer-based language models. The best performance was achieved with embeddings from a fine-tuned rubert-tiny2 model, yielding high value detection quality (F1 = 0.77, F1-macro = 0.83). This model provides a crucial step to a study of values within and between Russian social media users.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑