智能多模态搜索：Adobe Express中的上下文稀疏与稠密嵌入集成
Smart Multi-Modal Search: Contextual Sparse and Dense Embedding Integration in Adobe Express
AI总结：
本文提出一种集成稀疏、稠密嵌入与上下文特征的多模态搜索架构，通过AB测试优化Adobe Express模板搜索，显著降低空结果率超70%并提升点击率，增强了复杂查询相关性。
摘要：
随着用户内容和查询日益多模态化，对有效多模态搜索系统的需求不断增长。传统搜索系统通常依赖文本和元数据注释来索引图像，而像CLIP这样的多模态嵌入则允许直接使用文本和图像嵌入进行搜索。然而，基于嵌入的方法在整合用户区域设置和新近度等上下文特征时面临挑战。构建可扩展的多模态搜索系统需要微调多个组件。本文提出了一种多模态搜索架构，并进行了一系列AB测试，以优化Adobe Express模板搜索中的嵌入和多模态技术。我们探讨了嵌入模型选择、嵌入在匹配和排序中的作用，以及稠密与稀疏嵌入之间的平衡等考量。我们的迭代方法表明，利用稀疏、稠密和上下文特征能够增强短查询和长查询搜索，显著降低空结果率（超过70%），并提高点击率（CTR）。我们的研究结果为开发稳健的多模态搜索系统提供了见解，从而增强了复杂查询的相关性。
英文摘要：
As user content and queries become increasingly multi-modal, the need for effective multi-modal search systems has grown. Traditional search systems often rely on textual and metadata annotations for indexed images, while multi-modal embeddings like CLIP enable direct search using text and image embeddings. However, embedding-based approaches face challenges in integrating contextual features such as user locale and recency. Building a scalable multi-modal search system requires fine-tuning several components. This paper presents a multi-modal search architecture and a series of AB tests that optimize embeddings and multi-modal technologies in Adobe Express template search. We address considerations such as embedding model selection, the roles of embeddings in matching and ranking, and the balance between dense and sparse embeddings. Our iterative approach demonstrates how utilizing sparse, dense, and contextual features enhances short and long query search, significantly reduces null rates (over 70\%), and increases click-through rates (CTR). Our findings provide insights into developing robust multi-modal search systems, thereby enhancing relevance for complex queries.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
