用于高效图像探索和用户引导图像描述生成的视觉分析
Visual Analytics for Efficient Image Exploration and User-Guided Image Captioning
AI总结：
本文提出协同视觉分析系统，利用语言-图像模型生成的描述探索大规模图像数据集、识别偏差，并通过交互式界面评估和引导图像描述生成。
摘要：
预训练大规模语言-图像模型的最新进展开启了视觉理解的新时代，带来了重大飞跃。这些突破已被证明特别有助于解决以往令人畏惧的长期挑战。利用这些创新技术，本文解决视觉分析领域中两个众所周知的问题：（1）大规模图像数据集的高效探索以及其中潜在数据偏差的识别；（2）图像描述的评估及其生成过程的引导。一方面，通过可视化检查由语言-图像模型为图像数据集自动生成的描述，我们更深入地洞察视觉内容的语义基础，挖掘数据集中可能根深蒂固的数据偏差。另一方面，通过描绘视觉内容与文本描述之间的关联，我们揭示预训练语言-图像模型在描述能力方面的弱点，并提出一个交互式界面来引导描述生成。这两部分已合并为一个协同的视觉分析系统，促进视觉和文本元素的相互丰富。我们通过领域从业者以及针对大规模图像数据集的具体案例研究验证了该系统的有效性。
英文摘要：
Recent advancements in pre-trained large-scale language-image models have ushered in a new era of visual comprehension, offering a significant leap forward. These breakthroughs have proven particularly instrumental in addressing long-standing challenges that were previously daunting. Leveraging these innovative techniques, this paper tackles two well-known issues within the realm of visual analytics: (1) the efficient exploration of large-scale image datasets and identification of potential data biases within them; (2) the evaluation of image captions and steering of their generation process. On the one hand, by visually examining the captions automatically generated from language-image models for an image dataset, we gain deeper insights into the semantic underpinnings of the visual contents, unearthing data biases that may be entrenched within the dataset. On the other hand, by depicting the association between visual contents and textual captions, we expose the weaknesses of pre-trained language-image models in their captioning capability and propose an interactive interface to steer caption generation. The two parts have been coalesced into a coordinated visual analytics system, fostering mutual enrichment of visual and textual elements. We validate the effectiveness of the system with domain practitioners through concrete case studies with large-scale image datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
