arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.01782cs.CVcs.AI
收藏

解决深度视觉模型的一个根本性局限:缺乏空间注意力

Addressing a fundamental limitation in deep vision models: lack of spatial attention

Ali Borji

AI总结:

本文针对深度视觉模型缺乏空间注意力、处理整幅图像导致效率低下的问题,提出两种仅对变化区域进行选择性计算并更新输出图的方法,为构建更高效的下一代视觉模型提供新思路。

摘要:

本文的主要目的是强调当前深度学习模型,尤其是视觉模型中存在的一个显著局限。与人类视觉不同,人类视觉能够高效地仅选择必要的视觉区域进行进一步处理,从而实现高速度和低能耗,而深度视觉模型则处理整个图像。在这项工作中,我们从更广泛的视角审视这一问题,并提出了两种解决方案,可能为下一代更高效的视觉模型铺平道路。在第一种解决方案中,卷积和池化操作被选择性地应用于发生变化的区域,并将变化图发送到后续层。该变化图指示哪些计算需要重复执行。在第二种解决方案中,仅由语义分割模型处理修改过的区域,然后将得到的分割结果插入到先前输出图的相应区域中。代码可在 https://github.com/aliborji/spatial_attention 获取。

英文摘要:

The primary aim of this manuscript is to underscore a significant limitation in current deep learning models, particularly vision models. Unlike human vision, which efficiently selects only the essential visual areas for further processing, leading to high speed and low energy consumption, deep vision models process the entire image. In this work, we examine this issue from a broader perspective and propose two solutions that could pave the way for the next generation of more efficient vision models. In the first solution, convolution and pooling operations are selectively applied to altered regions, with a change map sent to subsequent layers. This map indicates which computations need to be repeated. In the second solution, only the modified regions are processed by a semantic segmentation model, and the resulting segments are inserted into the corresponding areas of the previous output map. The code is available at https://github.com/aliborji/spatial_attention.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑