arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2403.03134cs.CVcs.AIq-bio.NC
收藏

复杂中的简洁:利用深度分割模型解释视觉复杂度

Simplicity in Complexity : Explaining Visual Complexity using Deep Segmentation Models

Max Planck Institute for Biological Cybernetics(马克斯·普朗克生物控制论研究所);University of Tübingen(蒂宾根大学);Max Planck School of Cognition(马克斯·普朗克认知科学学院)

Tingke Shen, Surabhi S Nath, Aenne Brielmann, Peter Dayan

AI总结:

本文提出用SAM和FC-CLIP分割模型提取分割区域数与类别数两个特征,通过简单线性模型在六个多样化图像集上有效解释视觉复杂度,揭示图像复杂度本质上可能十分简单。

摘要:

视觉刺激的复杂度在许多认知现象中发挥着重要作用,包括注意力、参与度、可记忆性、时间感知和审美评价。尽管其重要性不言而喻,复杂度却一直未被充分理解,而颇具讽刺意味的是,以往的图像复杂度模型本身往往相当复杂。此前已有许多尝试寻找能够解释复杂度的手工设计特征,但这些特征通常依赖于特定数据集,因而难以泛化。另一方面,近期的工作采用深度神经网络来预测复杂度,但这些模型仍然难以解释,并且无法为问题的理论理解提供指导。本文提出利用基于分割的图像表示来建模复杂度。我们使用最先进的分割模型SAM和FC-CLIP,分别量化图像在多个粒度下的分割区域数量以及图像中的类别数量。我们发现,在六个包含自然场景和艺术图像的多样化图像集上,仅使用这两个特征的简单线性模型就能很好地解释复杂度。这表明图像的复杂度可能出人意料地简单。

英文摘要:

The complexity of visual stimuli plays an important role in many cognitive phenomena, including attention, engagement, memorability, time perception and aesthetic evaluation. Despite its importance, complexity is poorly understood and ironically, previous models of image complexity have been quite complex. There have been many attempts to find handcrafted features that explain complexity, but these features are usually dataset specific, and hence fail to generalise. On the other hand, more recent work has employed deep neural networks to predict complexity, but these models remain difficult to interpret, and do not guide a theoretical understanding of the problem. Here we propose to model complexity using segment-based representations of images. We use state-of-the-art segmentation models, SAM and FC-CLIP, to quantify the number of segments at multiple granularities, and the number of classes in an image respectively. We find that complexity is well-explained by a simple linear model with these two features across six diverse image-sets of naturalistic scene and art images. This suggests that the complexity of images can be surprisingly simple.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑