arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2401.05883cs.CL
收藏

面向社交媒体数据选择的生成式去重

Generative Deduplication For Socia Media Data Selection

The Hong Kong Polytechnic University(香港理工大学);Research Centre on Data Science & Artificial Intelligence(数据科学与人工智能研究中心)

Xianming Li, Jing Li

AI总结:

本文提出一种基于自监督生成模型的社交媒体数据去重框架,通过预测关键词捕获语义并加入时间维度高斯噪声,在减少训练样本的同时提升模型性能。

摘要:

社交媒体数据因其噪声特性而表现出严重的冗余,这导致处理过程中的训练时间增加和模型偏差。为解决这一问题,我们提出了一种新颖的生成式去重框架,通过移除语义重复的数据来进行社交媒体数据选择。虽然相关工作涉及任务特定训练中的数据选择,但我们的模型作为一种高效的预处理方法,能够普遍增强社交媒体自然语言处理流程。具体而言,我们通过自监督学习训练一个生成模型,预测关键词以捕获噪声社交媒体文本的语义,从而实现去重。同时,加入时间维度的高斯噪声以提高训练复杂度,避免学习到平凡特征。大量实验表明,与基线方法相比,我们的模型能够更好地减少训练样本,同时提升性能。结果显示了我们的模型在有效性和效率方面广泛推进社交媒体语言理解的潜力。

英文摘要:

Social media data exhibits severe redundancy caused by its noisy nature. It leads to increased training time and model bias in its processing. To address this issue, we propose a novel Generative Deduplication framework for social media data selection by removing semantically duplicate data. While related work involves data selection in task-specific training, our model acts as an efficient pre-processing method to universally enhance social media NLP pipelines. Specifically, we train a generative model via self-supervised learning to predict a keyword to capture the semantics of noisy social media text for deduplication. Meanwhile, time-dimensional Gaussian noise is added to improve training complexity and avoid learning trivial features. Extensive experiments suggest that our model can better reduce training samples while improving performance than baselines. The results show our model's potential to broadly advance social media language understanding in effectiveness and efficiency.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑