arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2405.07919cs.CV
收藏

探索图像超分辨率中的低通滤波行为

Exploring the Low-Pass Filtering Behavior in Image Super-Resolution

University of Electronic Science and Technology of China(电子科技大学)

Haoyu Deng, Zijing Xu, Yule Duan, Xiao Wu, Wenjie Shu, Liang-Jian Deng

AI总结:

本文利用信号处理理论解释图像超分辨率神经网络行为,发现sinc现象,提出HyRA方法揭示线性低通滤波与非线性高频注入机制,并设计FSDS指标量化高频信息。

摘要:

用于图像超分辨率(ISR)的深度神经网络相比插值等传统方法已展现出显著优势。然而,与具有坚实数学基础的传统方法相比,它们常被批评为“黑箱”。本文尝试利用信号处理领域的理论来解释深度神经网络在ISR中的行为。首先,我们报告了一个有趣的现象,称为“sinc现象”。当脉冲输入被馈入神经网络时,该现象会出现。然后,基于这一观察,我们提出了一种名为混合响应分析(HyRA)的方法,用于分析神经网络在ISR任务中的行为。具体而言,HyRA将神经网络分解为一个线性系统和一个非线性系统的并联连接,并证明线性系统起到低通滤波器的作用,而非线性系统注入高频信息。最后,为了量化注入的高频信息,我们为图像到图像任务引入了一个称为频谱分布相似度(FSDS)的指标。FSDS反映不同频率成分的分布相似性,能够捕捉传统指标可能忽略的细微差别。本文的代码、视频和原始实验结果可在以下地址找到:https://github.com/RisingEntropy/LPFInISR。

英文摘要:

Deep neural networks for image super-resolution (ISR) have shown significant advantages over traditional approaches like the interpolation. However, they are often criticized as 'black boxes' compared to traditional approaches with solid mathematical foundations. In this paper, we attempt to interpret the behavior of deep neural networks in ISR using theories from the field of signal processing. First, we report an intriguing phenomenon, referred to as `the sinc phenomenon.' It occurs when an impulse input is fed to a neural network. Then, building on this observation, we propose a method named Hybrid Response Analysis (HyRA) to analyze the behavior of neural networks in ISR tasks. Specifically, HyRA decomposes a neural network into a parallel connection of a linear system and a non-linear system and demonstrates that the linear system functions as a low-pass filter while the non-linear system injects high-frequency information. Finally, to quantify the injected high-frequency information, we introduce a metric for image-to-image tasks called Frequency Spectrum Distribution Similarity (FSDS). FSDS reflects the distribution similarity of different frequency components and can capture nuances that traditional metrics may overlook. Code, videos and raw experimental results for this paper can be found in: https://github.com/RisingEntropy/LPFInISR.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑