arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2512.04496cs.CV
收藏

移位窗口与双注意力机制:一种多模型架构用于镜面高光去除

Shift-Window Meets Dual Attention: A Multi-Model Architecture for Specular Highlight Removal

College of Mechanical and Vehicle Engineering, Chongqing University(重庆大学机械与车辆工程学院);College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院);Artificial Intelligence Center, Geely Automotive Research Institute(吉利汽车研究院人工智能中心)

Tianci Huo, Lingfeng Qi, Yuhan Chen, Qihong Xue, Jinyuan Shao, Hai Yu, Jie Li, Zhanhua Zhang, Guofa Li

AI总结:

本文提出多模型架构MM-SHR,结合卷积与注意力机制,有效去除不同尺度的镜面高光,提升视觉任务性能。

摘要:

实际环境中不可避免的镜面高光严重损害视觉性能,从而降低任务的有效性和效率。尽管存在大量专注于卷积神经网络模型局部信息或变压器模型全局信息的方法,但单一类型的模型在局部精细细节和全局长距离依赖性之间陷入建模困境,从而在不同尺度的镜面高光上表现不佳。因此,为了适应所有尺度的镜面高光,我们提出了一种用于镜面高光去除的多模型架构(MM-SHR),该架构能够有效捕捉高光区域的细粒度特征,并建模高光区域与非高光区域之间的长距离依赖性。具体来说,我们利用卷积操作在MM-SHR的浅层中提取局部细节,并利用注意力机制在深层中捕捉全局特征,确保操作效率和去除准确性。为了在不牺牲计算复杂性的情况下建模长距离依赖性,我们采用粗到细的方式,并提出Omni-Directional Attention Integration Block(OAIBlock)和Adaptive Region-Aware Hybrid-Domain Dual Attention Convolutional Network(HDDAConv),这些方法利用原始特征中的全方位像素位移和窗口分割操作来实现镜面高光去除。在三个基准任务和六种表面材料上的大量实验结果表明,MM-SHR在镜面高光去除的准确性和效率方面均优于最先进的方法。该实现将在https://github.com/Htcicv/MM-SHR上公开。

英文摘要:

Inevitable specular highlights in practical environments severely impair the visual performance, thus degrading the task effectiveness and efficiency. Although there exist considerable methods that focus on local information from convolutional neural network models or global information from transformer models, the single-type model falls into a modeling dilemma between local fine-grained details and global long-range dependencies, thus deteriorating for specular highlights with different scales. Therefore, to accommodate specular highlights of all scales, we propose a multi-model architecture for specular highlight removal (MM-SHR) that effectively captures fine-grained features in highlight regions and models long-range dependencies between highlight and highlight-free areas. Specifically, we employ convolution operations to extract local details in the shallow layers of MM-SHR, and utilize the attention mechanism to capture global features in the deep layers, ensuring both operation efficiency and removal accuracy. To model long-range dependencies without compromising computational complexity, we utilize a coarse-to-fine manner and propose Omni-Directional Attention Integration Block(OAIBlock) and Adaptive Region-Aware Hybrid-Domain Dual Attention Convolutional Network(HDDAConv) , which leverage omni-directiona pixel-shifting and window-dividing operations at the raw features to achieve specular highlight removal. Extensive experimental results on three benchmark tasks and six types of surface materials demonstrate that MM-SHR outperforms state-of-the-art methods in both accuracy and efficiency for specular highlight removal. The implementation will be made publicly available at https://github.com/Htcicv/MM-SHR.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑