InfoCIR：面向复合图像检索的多媒体分析
InfoCIR: Multimedia Analysis for Composed Image Retrieval
AI总结：
InfoCIR通过整合检索、可解释性与提示工程，提供交互式界面，帮助用户理解多模态提示与嵌入空间的交互，提升模型开发中的洞察力生成。
摘要：
复合图像检索（CIR）允许用户通过结合参考图像和描述所需修改的文本提示来搜索图像。尽管视觉语言模型如CLIP通过将多种模态嵌入到联合空间中流行化了此任务，但开发者仍然缺乏揭示这些多模态提示如何与嵌入空间交互以及为何微小用词变化会显著改变结果的工具。我们提出了InfoCIR，一个视觉分析系统，通过在单个交互式仪表板中耦合检索、可解释性和提示工程来填补这一空白。InfoCIR集成了最先进的CIR后端（SEARLE arXiv:2303.15247）与一个六面板界面，该界面（i）允许用户组合图像+文本查询，（ii）将前k结果投影到低维空间中使用统一流形近似和投影（UMAP）以进行空间推理，（iii）叠加基于相似性的显著性图和基于梯度的token- attribution条形图以进行局部解释，（iv）利用LLM驱动的提示增强器生成反事实变体并可视化这些变化如何影响用户选定目标图像的排名。基于Plotly-Dash构建的模块化架构允许新模型、数据集和属性方法以最小的努力进行插拔。我们主张InfoCIR有助于诊断检索失败、指导提示增强，并在模型开发过程中加速洞察生成。所有允许重现演示的源代码可在https://github.com/giannhskp/InfoCIR上获得。
英文摘要：
Composed Image Retrieval (CIR) allows users to search for images by combining a reference image with a text prompt that describes desired modifications. While vision-language models like CLIP have popularized this task by embedding multiple modalities into a joint space, developers still lack tools that reveal how these multimodal prompts interact with embedding spaces and why small wording changes can dramatically alter the results. We present InfoCIR, a visual analytics system that closes this gap by coupling retrieval, explainability, and prompt engineering in a single, interactive dashboard. InfoCIR integrates a state-of-the-art CIR back-end (SEARLE arXiv:2303.15247) with a six-panel interface that (i) lets users compose image + text queries, (ii) projects the top-k results into a low-dimensional space using Uniform Manifold Approximation and Projection (UMAP) for spatial reasoning, (iii) overlays similarity-based saliency maps and gradient-derived token-attribution bars for local explanation, and (iv) employs an LLM-powered prompt enhancer that generates counterfactual variants and visualizes how these changes affect the ranking of user-selected target images. A modular architecture built on Plotly-Dash allows new models, datasets, and attribution methods to be plugged in with minimal effort. We argue that InfoCIR helps diagnose retrieval failures, guides prompt enhancement, and accelerates insight generation during model development. All source code allowing for a reproducible demo is available at https://github.com/giannhskp/InfoCIR.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
