面向短视频幽默检测的基于对比预训练的评论辅助视频-语言对齐方法
Comment-aided Video-Language Alignment via Contrastive Pre-training for Short-form Video Humor Detection
AI总结：
针对短视频幽默检测任务，提出基于数据增强多模态对比预训练的评论辅助视频-语言对齐双分支层次模型，在两个数据集上性能超越现有最优方法及基线，相关资源已开源。
摘要：
多模态幽默检测在情感计算中日益重要，这与短视频分享在社交媒体平台上的影响力不断扩大相关。本文提出了一种用于短视频幽默检测（SVHD）的新型双分支层次模型，名为评论辅助视频-语言对齐（CVLA），该模型通过数据增强的多模态对比预训练实现。值得注意的是，我们的CVLA不仅处理跨不同模态通道的原始信号，还通过在一致语义空间中对齐视频和语言组件，生成恰当的多模态表示。在DY11k和UR-FUNNY两个幽默检测数据集上的实验结果表明，CVLA的性能显著优于现有最优方法及多个有竞争力的基线方法。我们的数据集、代码和模型发布于https://github.com/yliu-cs/CVLA。
英文摘要：
The growing importance of multi-modal humor detection within affective computing correlates with the expanding influence of short-form video sharing on social media platforms. In this paper, we propose a novel two-branch hierarchical model for short-form video humor detection (SVHD), named Comment-aided Video-Language Alignment (CVLA) via data-augmented multi-modal contrastive pre-training. Notably, our CVLA not only operates on raw signals across various modal channels but also yields an appropriate multi-modal representation by aligning the video and language components within a consistent semantic space. The experimental results on two humor detection datasets, including DY11k and UR-FUNNY, demonstrate that CVLA dramatically outperforms state-of-the-art and several competitive baseline approaches. Our dataset, code and model release at https://github.com/yliu-cs/CVLA.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
