ContextIQ：一种用于上下文广告的基于多模态专家的视频检索系统
ContextIQ: A Multimodal Expert-Based Video Retrieval System for Contextual Advertising
AI总结：
本文提出ContextIQ，一种专为上下文广告设计的基于多模态专家的视频检索系统，通过利用视频、音频、转录文本和元数据等模态专家创建语义丰富的表示，无需联合训练即可在多个基准上取得优异表现，并解决品牌安全等问题。
摘要：
上下文广告提供与用户正在观看的内容相匹配的广告。社交媒体和流媒体服务上视频内容的快速增长，加上对隐私的担忧，增加了对上下文广告的需求。在正确的上下文中投放合适的广告能创造无缝且愉悦的广告观看体验，从而带来更高的受众参与度，最终实现更好的广告变现。从技术角度来看，有效的上下文广告需要一个能够在非常细粒度的层面上理解复杂视频内容的视频检索系统。当前基于联合多模态训练的文本到视频检索模型需要大型数据集和大量计算资源，这限制了它们的实用性，并且缺乏广告生态系统集成所需的关键功能。我们介绍了ContextIQ，这是一个专门为上下文广告设计的基于多模态专家的视频检索系统。ContextIQ利用特定模态的专家——视频、音频、转录文本（字幕）以及元数据（如对象、动作、情感等）——来创建语义丰富的视频表示。我们表明，我们的系统在无需联合训练的情况下，在多个文本到视频检索基准上取得了优于或可比的最先进模型和商业解决方案的结果。我们的消融研究强调了利用多种模态来提高视频检索准确性的好处，而不是单独使用视觉-语言模型。此外，我们展示了像ContextIQ这样的视频检索系统如何在广告生态系统中用于上下文广告，同时解决与品牌安全和过滤不当内容相关的问题。
英文摘要：
Contextual advertising serves ads that are aligned to the content that the user is viewing. The rapid growth of video content on social platforms and streaming services, along with privacy concerns, has increased the need for contextual advertising. Placing the right ad in the right context creates a seamless and pleasant ad viewing experience, resulting in higher audience engagement and, ultimately, better ad monetization. From a technology standpoint, effective contextual advertising requires a video retrieval system capable of understanding complex video content at a very granular level. Current text-to-video retrieval models based on joint multimodal training demand large datasets and computational resources, limiting their practicality and lacking the key functionalities required for ad ecosystem integration. We introduce ContextIQ, a multimodal expert-based video retrieval system designed specifically for contextual advertising. ContextIQ utilizes modality-specific experts-video, audio, transcript (captions), and metadata such as objects, actions, emotion, etc.-to create semantically rich video representations. We show that our system, without joint training, achieves better or comparable results to state-of-the-art models and commercial solutions on multiple text-to-video retrieval benchmarks. Our ablation studies highlight the benefits of leveraging multiple modalities for enhanced video retrieval accuracy instead of using a vision-language model alone. Furthermore, we show how video retrieval systems such as ContextIQ can be used for contextual advertising in an ad ecosystem while also addressing concerns related to brand safety and filtering inappropriate content.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
