VPN:用于鲁棒内容归属的视频来源网络
VPN: Video Provenance Network for Robust Content Attribution
摘要:
我们提出了VPN -一种内容归属方法,用于从在线共享的视频中恢复出处信息。平台和用户经常将视频转换为不同的质量,编解码器,大小,形状等,或者稍微编辑其内容,例如添加文本或表情符号,因为它们在网上重新分发。我们学习了一个强大的搜索嵌入匹配这样的视频,这些变换不变,使用全长或截断的视频查询。一旦与视频剪辑的可信数据库匹配,就向用户呈现关于剪辑的来源的相关联的信息。我们使用一个倒排索引来匹配视频的时间块,使用后期融合来结合视觉和音频特征。在这两种情况下,特征都是通过使用对比学习在原始和增强视频剪辑的数据集上训练的深度神经网络来提取的。我们在10万个视频的语料库中展示了高准确率的召回率。
英文摘要:
We present VPN - a content attribution method for recovering provenance information from videos shared online. Platforms, and users, often transform video into different quality, codecs, sizes, shapes, etc. or slightly edit its content such as adding text or emoji, as they are redistributed online. We learn a robust search embedding for matching such video, invariant to these transformations, using full-length or truncated video queries. Once matched against a trusted database of video clips, associated information on the provenance of the clip is presented to the user. We use an inverted index to match temporal chunks of video using late-fusion to combine both visual and audio features. In both cases, features are extracted via a deep neural network trained using contrastive learning on a dataset of original and augmented video clips. We demonstrate high accuracy recall over a corpus of 100,000 videos.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。