文本引导如何影响视觉注意力？数据库与模型
How is Visual Attention Influenced by Text Guidance? Database and Model
AI总结：
针对文本引导对视觉注意力的影响研究不足的问题，构建含1200组样本的SJTU-TIS数据库与基准，提出融合图文特征的TGSal显著性预测模型，性能优于现有最优模型。
摘要：
视觉注意力的分析与预测一直是计算机视觉和图像处理领域的核心任务。在实际应用中，图像通常伴随各类文本描述，但鲜有研究探讨文本描述对视觉注意力的影响，更未开发出考虑文本引导的视觉显著性预测模型。本文从主观和客观两个维度对文本引导图像显著性（text-guided image saliency, TIS）展开全面研究。具体而言，我们构建了名为SJTU-TIS的TIS数据库，包含1200组文本-图像对及对应采集的眼动数据。基于搭建的SJTU-TIS数据库，我们分析了不同文本描述对视觉注意力的影响。随后，为推动考虑文本影响的显著性预测模型发展，我们采用当前最优的显著性模型为SJTU-TIS数据库构建了基准。最后，鉴于现有多数显著性模型忽略文本描述对视觉注意力的影响，我们进一步提出文本引导显著性（text-guided saliency, TGSal）预测模型，该模型提取并融合图像特征与文本特征，以预测不同文本描述条件下的图像显著性。我们提出的模型在SJTU-TIS数据库和纯图像显著性数据库上，各项评估指标均显著优于当前最优显著性模型。SJTU-TIS数据库及所提TGSal模型的代码将发布于：https://github.com/IntMeGroup/TGSal。
英文摘要：
The analysis and prediction of visual attention have long been crucial tasks in the fields of computer vision and image processing. In practical applications, images are generally accompanied by various text descriptions, however, few studies have explored the influence of text descriptions on visual attention, let alone developed visual saliency prediction models considering text guidance. In this paper, we conduct a comprehensive study on text-guided image saliency (TIS) from both subjective and objective perspectives. Specifically, we construct a TIS database named SJTU-TIS, which includes 1200 text-image pairs and the corresponding collected eye-tracking data. Based on the established SJTU-TIS database, we analyze the influence of various text descriptions on visual attention. Then, to facilitate the development of saliency prediction models considering text influence, we construct a benchmark for the established SJTU-TIS database using state-of-the-art saliency models. Finally, considering the effect of text descriptions on visual attention, while most existing saliency models ignore this impact, we further propose a text-guided saliency (TGSal) prediction model, which extracts and integrates both image features and text features to predict the image saliency under various text-description conditions. Our proposed model significantly outperforms the state-of-the-art saliency models on both the SJTU-TIS database and the pure image saliency databases in terms of various evaluation metrics. The SJTU-TIS database and the code of the proposed TGSal model will be released at: https://github.com/IntMeGroup/TGSal.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
