EviRank：用于多模态图像重排序的结构化相关性证据

中文摘要 AI 辅助

现实世界的图像搜索查询是多模态且具有组合性的：“找到这件粉色衬衫”指定了要保留的实体、要修改的属性以及要忽略的上下文。然而现有的重排序器要么将这种多方面的相关性压缩为不透明的嵌入，要么依赖自由形式的思维链，很容易遗漏或虚构细粒度约束。借鉴NLP中基于评分标准和检查表的评估方法，我们将多模态图像重排序重新定义为语义约束满足问题，并提出了EviRank，它可将任何查询——纯文本、纯图像或组合查询——解析为统一的证据包：跨六个语义槽（如实体、属性、关系）的类型化标准，每个标准被标记为必需、禁止或可忽略。重排序随后简化为证据条件下的验证，在单一无训练过程中结合确定性评分标准打分和基于证据的列表级比较。这种显式证据还可作为结构化监督，用于可选地蒸馏出轻量级学生模型。在涵盖文本到图像、图像到图像和组合图像检索的五个基准测试中，EviRank取得了最先进的性能，且蒸馏出的学生模型以显著更低的成本保留了教师模型超过90%的能力。

英文摘要

Real-world image search queries are multimodal and compositional: ``find this shirt in pink'' specifies an entity to retain, an attribute to modify, and context to ignore. Yet existing re-rankers either compress such multifaceted relevance into an opaque embedding or rely on free-form chain-of-thought that easily omits or hallucinates fine-grained constraints. Drawing on rubric- and checklist-based evaluation from NLP, we recast multimodal image re-ranking as a semantic constraint satisfaction problem and propose EviRank, which parses any query - text-only, image-only, or composed - into a unified evidence package: typed criteria across six semantic slots (e.g., entities, attributes, relations), each labelled required, forbidden, or ignorable. Re-ranking then reduces to evidence-conditioned verification, combining deterministic rubric scoring and evidence-grounded listwise comparison in a single training-free procedure. The explicit evidence can further serve as structured supervision for optionally distilling a lightweight student. Across five benchmarks spanning text-to-image, image-to-image, and composed image retrieval, EviRank achieves state-of-the-art performance, and the distilled student preserves over 90% of the teacher's capability at substantially lower cost.

发表机构

• The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
• Tencent Yuanbao（腾讯元宝）
• The University of Hong Kong（香港大学）

机构由 AI 辅助整理，请以论文原文为准。