ProbeScout：面向属性引导图像搜索的可视分析系统
ProbeScout: Visual Analytics for Attribute-Guided Image Search
- The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
- Data Science and Analytics Thrust, Information Hub, The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州）信息枢纽数据科学与分析 thrust）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
ProbeScout是一个可视分析系统，通过可组合属性探针和融合排序支持属性引导的图像搜索，在少量标注下提升检索性能并支持交互式细化。
AI中文摘要：
分析人员通常需要识别同时满足多个视觉条件的图像，例如黄昏时分带有交通信号灯的十字路口，用于模型诊断、数据集整理和定向训练。基于嵌入的检索可以高效地对大型图库进行排序，但视觉上占主导地位的条件可能会掩盖较弱的条件，并且单一的相似度分数无法强制执行所需的合取关系。视觉问答（VQA）可以显式地验证条件，然而对完整图库进行穷举式应用成本高昂，尤其是在分析人员细化查询时。这些局限性促使我们在属性层面将人类纳入循环，使分析人员能够快速为每个条件构建证据，并在请求变化时重用这些证据。因此，我们提出了ProbeScout，一个支持此循环的可视分析系统。它首先从稀疏的VQA标签中构建可组合的属性探针，并将其融合为感知合取关系的初始排序。协调视图通过过滤和组合这些探针输出来支持快速筛选、近似遗漏诊断和即时子集构建。分析人员提供轻量级的属性和查询级反馈，驱动融合权重的分阶段细化，同时保持探针固定。这些经过验证的属性可复用于未来的查询。我们在三个数据集的17个检索任务上评估了ProbeScout，显示出相对于嵌入基线的检索性能提升。一项单独的10任务比较在标注最多2%的图库图像的情况下，实现了比穷举式VQA更高的任务宏平均精度（AP）和F1分数。两个案例研究进一步展示了ProbeScout如何在现实工作流程中支持交互式分析和细化。
英文摘要：
Analysts often need to identify images that jointly satisfy multiple visual conditions, such as a crossroads with traffic lights at dusk, for model diagnosis, dataset curation, and targeted training. Embedding-based retrieval can rank the large gallery efficiently, but a visually dominant condition can obscure weaker conditions, and a single similarity score does not enforce the required conjunction. Visual question answering (VQA) can explicitly verify conditions, yet exhaustively applying it to the full gallery is costly, especially when analysts refine their query. These limitations motivate keeping humans in the loop at the attribute level, where analysts can quickly build evidence for each condition and reuse it when the request changes. We therefore present ProbeScout, a visual analytics system that supports this loop. It first builds composable attribute probes from sparse VQA labels and fuses them into a conjunction-aware initial ranking. Coordinated views support rapid screening, near-miss diagnosis, and on-the-fly subset construction by filtering and combining these probe outputs. Analysts provide lightweight attribute- and query-level feedback, which drives staged refinement of fusion weights while keeping the probes fixed. These verified attributes can be reused for future queries. We evaluate ProbeScout on 17 retrieval tasks across three datasets, showing improved retrieval over embedding baselines. A separate 10-task comparison achieves higher task-macro AP and F1 than exhaustive VQA while labeling at most 2% of the gallery images. Two case studies further demonstrate how ProbeScout supports interactive analysis and refinement in realistic workflows.
