面向内容审核中缺失模态推理的多模态引导网络
Multimodal Guidance Network for Missing-Modality Inference in Content Moderation
AI总结：
针对内容审核中推理阶段模态缺失问题，本文提出多模态引导网络，利用多模态表示训练更优的单模态模型，在暴力检测中显著提升性能且不增加推理计算成本。
摘要：
多模态深度学习，尤其是视觉-语言模型，近年来获得了显著发展，极大地提升了包括内容审核和暴力检测在内的许多下游任务的性能。然而，标准的多模态方法通常假设训练和推理阶段的模态保持一致，这限制了其在许多真实世界场景中的应用，因为在推理阶段某些模态可能不可用。尽管现有研究通过重建缺失模态来缓解这一问题，但它们不可避免地增加了不必要的计算成本，而这对于工业界大规模部署的基础设施而言可能同样关键。为此，我们提出了一种新颖的引导网络，在训练过程中促进知识共享，利用多模态表示来训练更好的单模态模型以用于推理。在暴力检测中的真实世界实验表明，我们提出的框架训练出的单模态模型显著优于传统方式训练的对应模型，同时避免了推理阶段计算成本的增加。
英文摘要：
Multimodal deep learning, especially vision-language models, have gained significant traction in recent years, greatly improving performance on many downstream tasks, including content moderation and violence detection. However, standard multimodal approaches often assume consistent modalities between training and inference, limiting applications in many real-world use cases, as some modalities may not be available during inference. While existing research mitigates this problem through reconstructing the missing modalities, they unavoidably increase unnecessary computational cost, which could be just as critical, especially for large, deployed infrastructures in industry. To this end, we propose a novel guidance network that promotes knowledge sharing during training, taking advantage of the multimodal representations to train better single-modality models to be used for inference. Real-world experiments in violence detection shows that our proposed framework trains single-modality models that significantly outperform traditionally trained counterparts, while avoiding increases in computational cost for inference.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
