arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2407.12255cs.CV
收藏

双混合注意力网络用于高光去除

Dual-Hybrid Attention Network for Specular Highlight Removal

University of Macau(澳门大学);Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院);Baoshan University(保山学院);Huizhou University(惠州学院)

Xiaojiao Guo, Xuhang Chen, Shenghong Luo, Shuqiang Wang, Chi-Man Pun

AI总结:

本文提出双混合注意力网络DHAN-SHR,无需额外先验或监督,通过局部与全局混合注意力机制有效去除高光,并在大规模基准上超越18种现有方法。

摘要:

高光去除在多媒体应用中扮演着关键角色,因为它提高了图像和视频的质量和可解释性,最终提升了诸如基于内容的检索、物体识别和场景理解等下游任务的性能。尽管基于深度学习的方法取得了显著进展,但目前最先进的方法往往依赖于额外的先验或监督,限制了其实用性和泛化能力。在本文中,我们提出了用于高光去除的双混合注意力网络(DHAN-SHR),这是一个端到端的网络,引入了新颖的混合注意力机制,能够有效地在不同尺度和域之间捕获和处理信息,而无需依赖额外的先验或监督。DHAN-SHR由两个关键组件组成:自适应局部混合域双注意力变换器(L-HD-DAT)和自适应全局双注意力变换器(G-DAT)。L-HD-DAT捕获局部通道间和像素间的依赖关系,同时整合频谱域特征,使网络能够有效地建模高光与底层表面属性之间的复杂交互。G-DAT建模全局通道间关系和长距离像素依赖,使网络能够将上下文信息传播到整个图像,并生成更连贯一致的无高光结果。为了评估DHAN-SHR的性能并促进该领域的未来研究,我们编译了一个大规模基准数据集,包含具有不同高光水平的多样化图像。通过大量实验,我们证明DHAN-SHR在定量和定性上均优于18种最先进的方法,为多媒体应用中的高光去除设立了新标准。

英文摘要:

Specular highlight removal plays a pivotal role in multimedia applications, as it enhances the quality and interpretability of images and videos, ultimately improving the performance of downstream tasks such as content-based retrieval, object recognition, and scene understanding. Despite significant advances in deep learning-based methods, current state-of-the-art approaches often rely on additional priors or supervision, limiting their practicality and generalization capability. In this paper, we propose the Dual-Hybrid Attention Network for Specular Highlight Removal (DHAN-SHR), an end-to-end network that introduces novel hybrid attention mechanisms to effectively capture and process information across different scales and domains without relying on additional priors or supervision. DHAN-SHR consists of two key components: the Adaptive Local Hybrid-Domain Dual Attention Transformer (L-HD-DAT) and the Adaptive Global Dual Attention Transformer (G-DAT). The L-HD-DAT captures local inter-channel and inter-pixel dependencies while incorporating spectral domain features, enabling the network to effectively model the complex interactions between specular highlights and the underlying surface properties. The G-DAT models global inter-channel relationships and long-distance pixel dependencies, allowing the network to propagate contextual information across the entire image and generate more coherent and consistent highlight-free results. To evaluate the performance of DHAN-SHR and facilitate future research in this area, we compile a large-scale benchmark dataset comprising a diverse range of images with varying levels of specular highlights. Through extensive experiments, we demonstrate that DHAN-SHR outperforms 18 state-of-the-art methods both quantitatively and qualitatively, setting a new standard for specular highlight removal in multimedia applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑