arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2601.04720cs.CL
收藏

Qwen3-VL-Embedding 和 Qwen3-VL-Reranker:面向最新多模态检索与排序的统一框架

Qwen3-VL-Embedding and Qwen3-VL-Reranker: A Unified Framework for State-of-the-Art Multimodal Retrieval and Ranking

Tongyi Lab, Alibaba Group(阿里巴巴集团通义实验室)

Mingxin Li, Yanzhao Zhang, Dingkun Long, Keqin Chen, Sibo Song, Shuai Bai, Zhibo Yang, Pengjun Xie, An Yang, Dayiheng Liu, Jingren Zhou, Junyang Lin

AI总结:

Qwen3-VL-Embedding和Qwen3-VL-Reranker通过统一框架实现多模态检索与排序的高精度性能。

摘要:

本文介绍了Qwen3-VL-Embedding和Qwen3-VL-Reranker模型系列,这是基于Qwen3-VL基础模型构建的Qwen家族最新扩展。二者共同提供了一个端到端的流程,通过将文本、图像、文档图像和视频等多样化模态映射到统一的表示空间中,实现高精度的多模态搜索。Qwen3-VL-Embedding模型采用多阶段训练范式,从大规模对比预训练到重排序模型蒸馏,生成语义丰富且高维的向量。它支持Matryoshka表示学习,能够灵活调整嵌入维度,并处理高达32k个标记的输入。此外,Qwen3-VL-Reranker通过具有交叉注意力机制的交叉编码器架构,对查询-文档对进行细粒度的相关性估计。两个模型系列继承了Qwen3-VL的多语言能力,支持超过30种语言,并以2B和8B参数规模发布,以适应不同的部署需求。实验评估表明,Qwen3-VL-Embedding系列在多种多模态嵌入评估基准上取得了最先进的结果。具体而言,Qwen3-VL-Embedding-8B在MMEB-V2上的总体得分为77.8,在所有模型中排名第一(截至2025年1月8日)。本文介绍了该系列的架构、训练方法和实际能力,展示了其在各种多模态检索任务中的有效性,包括图像-文本检索、视觉问答和视频-文本匹配。

英文摘要:

In this report, we introduce the Qwen3-VL-Embedding and Qwen3-VL-Reranker model series, the latest extensions of the Qwen family built on the Qwen3-VL foundation model. Together, they provide an end-to-end pipeline for high-precision multimodal search by mapping diverse modalities, including text, images, document images, and video, into a unified representation space. The Qwen3-VL-Embedding model employs a multi-stage training paradigm, progressing from large-scale contrastive pre-training to reranking model distillation, to generate semantically rich high-dimensional vectors. It supports Matryoshka Representation Learning, enabling flexible embedding dimensions, and handles inputs up to 32k tokens. Complementing this, Qwen3-VL-Reranker performs fine-grained relevance estimation for query-document pairs using a cross-encoder architecture with cross-attention mechanisms. Both model series inherit the multilingual capabilities of Qwen3-VL, supporting more than 30 languages, and are released in $\textbf{2B}$ and $\textbf{8B}$ parameter sizes to accommodate diverse deployment requirements. Empirical evaluations demonstrate that the Qwen3-VL-Embedding series achieves state-of-the-art results across diverse multimodal embedding evaluation benchmarks. Specifically, Qwen3-VL-Embedding-8B attains an overall score of $\textbf{77.8}$ on MMEB-V2, ranking first among all models (as of January 8, 2025). This report presents the architecture, training methodology, and practical capabilities of the series, demonstrating their effectiveness on various multimodal retrieval tasks, including image-text retrieval, visual question answering, and video-text matching.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑