arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.07307cs.CV
收藏

面向显著性预测的基于潜在扩散的数据增强方法

Data Augmentation via Latent Diffusion for Saliency Prediction

School of Computer and Communication Sciences, EPFL(洛桑联邦理工学院计算机与通信科学学院)

Bahar Aydemir, Deblina Bhattacharjee, Tong Zhang, Mathieu Salzmann, Sabine Süsstrunk

AI总结:

针对显著性预测标注数据不足问题,提出融合光度与语义属性、引入显著性引导交叉注意力的潜在扩散数据增强方法,有效提升多类模型在公开基准上的性能,预测结果与人类注意力模式高度吻合。

摘要:

显著性预测模型受限于标注数据的多样性与数量不足。旋转、裁剪等标准数据增强技术会改变场景构成,进而影响显著性。我们提出一种适用于深度显著性预测的新型数据增强方法,可在编辑自然图像的同时保留真实场景的复杂性与多样性。由于显著性依赖于高层与低层特征,我们的方法通过融合颜色、对比度、亮度、类别等光度与语义属性来学习这两类特征。为此,我们引入了显著性引导的交叉注意力机制,可对光度属性进行针对性编辑,从而增强特定图像区域内的显著性。实验结果表明,我们的数据增强方法持续提升了多种显著性模型的性能。此外,将增强特征用于显著性预测,在公开的显著性基准测试集上取得了更优的性能。用户研究验证,我们的预测结果与编辑后图像中的人类视觉注意力模式高度吻合。

英文摘要:

Saliency prediction models are constrained by the limited diversity and quantity of labeled data. Standard data augmentation techniques such as rotating and cropping alter scene composition, affecting saliency. We propose a novel data augmentation method for deep saliency prediction that edits natural images while preserving the complexity and variability of real-world scenes. Since saliency depends on high-level and low-level features, our approach involves learning both by incorporating photometric and semantic attributes such as color, contrast, brightness, and class. To that end, we introduce a saliency-guided cross-attention mechanism that enables targeted edits on the photometric properties, thereby enhancing saliency within specific image regions. Experimental results show that our data augmentation method consistently improves the performance of various saliency models. Moreover, leveraging the augmentation features for saliency prediction yields superior performance on publicly available saliency benchmarks. Our predictions align closely with human visual attention patterns in the edited images, as validated by a user study.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑