E-VAds:面向多模态大语言模型的电商短视频理解基准
E-VAds: An E-commerce Short Videos Understanding Benchmark for MLLMs
AI总结:
提出电商短视频理解基准E-VAds,通过多模态信息密度评估框架量化领域复杂性,并构建多智能体生成的问答数据集,最后开发基于强化学习的推理模型E-VAds-R1,在商业意图推理上实现109.2%的性能提升。
摘要:
电商短视频代表了在线视频行业中高收入的细分领域,其特点是目标驱动的格式和密集的多模态信号。当前模型通常难以处理这些视频,因为现有基准主要关注通用任务,忽略了商业意图的推理。在这项工作中,我们首先提出了一个多模态信息密度评估框架,以量化该领域的复杂性。我们的评估显示,与主流数据集相比,电商内容在视觉、音频和文本模态上表现出显著更高的密度,为视频理解建立了更具挑战性的前沿。为了弥补这一差距,我们引入了电商视频广告基准(E-VAds),这是首个专门为电商短视频理解设计的基准。我们从淘宝精选了3,961个高质量视频,涵盖广泛的产品类别,并使用多智能体系统生成了19,785个开放式问答对。这些问题被组织成两个主要维度,即感知与认知和推理,包含五个不同的任务。最后,我们开发了E-VAds-R1,一个基于强化学习的推理模型,具有称为MG-GRPO的多粒度奖励设计。该策略为早期探索提供平滑指导,同时为专家级精度创造非线性激励。实验结果表明,E-VAds-R1在仅使用几百个训练样本的情况下,在商业意图推理上实现了109.2%的性能提升。
英文摘要:
E-commerce short videos represent a high-revenue segment of the online video industry characterized by a goal-driven format and dense multi-modal signals. Current models often struggle with these videos because existing benchmarks focus primarily on general-purpose tasks and neglect the reasoning of commercial intent. In this work, we first propose a multi-modal information density assessment framework to quantify the complexity of this domain. Our evaluation reveals that e-commerce content exhibits substantially higher density across visual, audio, and textual modalities compared to mainstream datasets, establishing a more challenging frontier for video understanding. To address this gap, we introduce E-commerce Video Ads Benchmark, which is the first benchmark specifically designed for e-commerce short video understanding. We curated 3,961 high-quality videos from Taobao covering a wide range of product categories and used a multi-agent system to generate 19,785 open-ended Q&A pairs, which consist of five distinct tasks. Finally, we develop E-VAds-R1, an RL-based reasoning model featuring a multi-grained reward design called MG-GRPO. This strategy provides smooth guidance for early exploration while creating a non-linear incentive for expert-level precision. Experimental results demonstrate that E-VAds-R1 achieves a 109.2% performance gain in commercial intent reasoning with only a few hundred training samples. Data is available at https://github.com/TaobaoTmall-AlgorithmProducts/E-VAds_Benchmark.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。