arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2304.08401cs.CVcs.LG
收藏

基于对比学习的多模态短视频谣言检测系统

Multimodal Short Video Rumor Detection System Based on Contrastive Learning

Sichuan University Cyberspace Security Academy(四川大学网络空间安全学院)

Yuxing Yang, Junhao Zhao, Siyi Wang, Xiangyu Min, Pengchao Wang, Haizhou Wang

AI总结:

提出基于对比学习的多模态短视频谣言检测系统,融合视频与文本特征并引入外部知识,有效解决短视频谣言识别难题。

摘要:

随着短视频平台作为新闻传播的主要渠道的兴起,中国的主要平台已逐渐成为虚假新闻滋生的肥沃土壤。然而,由于信息量巨大以及视频间存在共享特征导致同质化,区分短视频谣言面临重大挑战。为了有效应对短视频谣言的传播,我们的研究小组提出了一种综合多模态特征融合和外部知识整合的方法,综合考虑了每种算法的优缺点。所提出的检测方法包括以下步骤:(1)创建一个包含从短视频中提取的多个特征的综合数据集;(2)开发一个多模态谣言检测模型:首先,我们采用时序分段网络(TSN)视频编码模型提取视频特征,随后利用光学字符识别(OCR)和自动语音识别(ASR)提取文本特征。接着,采用BERT模型融合文本和视频特征;(3)通过对比学习实现区分:我们通过爬取相关来源获取外部知识,并利用向量数据库将这些知识整合到分类输出中。我们的研究过程以实际考虑为导向,从本研究中获得的知识在实际场景中具有重要价值,如短视频谣言识别和社会舆论管理。

英文摘要:

With the rise of short video platforms as prominent channels for news dissemination, major platforms in China have gradually evolved into fertile grounds for the proliferation of fake news. However, distinguishing short video rumors poses a significant challenge due to the substantial amount of information and shared features among videos, resulting in homogeneity. To address the dissemination of short video rumors effectively, our research group proposes a methodology encompassing multimodal feature fusion and the integration of external knowledge, considering the merits and drawbacks of each algorithm. The proposed detection approach entails the following steps: (1) creation of a comprehensive dataset comprising multiple features extracted from short videos; (2) development of a multimodal rumor detection model: first, we employ the Temporal Segment Networks (TSN) video coding model to extract video features, followed by the utilization of Optical Character Recognition (OCR) and Automatic Speech Recognition (ASR) to extract textual features. Subsequently, the BERT model is employed to fuse textual and video features; (3) distinction is achieved through contrast learning: we acquire external knowledge by crawling relevant sources and leverage a vector database to incorporate this knowledge into the classification output. Our research process is driven by practical considerations, and the knowledge derived from this study will hold significant value in practical scenarios, such as short video rumor identification and the management of social opinions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑