从美学、失真与显著性评估UHD图像质量
Assessing UHD Image Quality from Aesthetics, Distortions, and Saliency
AI总结：
针对UHD图像质量评估的计算复杂度和细节损失问题，提出多分支DNN从美学、失真和显著性三视角提取特征，并在UHD-IQA上以最低计算复杂度取得最佳性能。
摘要：
UHD图像通常具有等于或高于4K的分辨率，对高效的图像质量评估（IQA）算法构成重大挑战，因为采用全分辨率图像作为输入会导致极高的计算复杂度，而常用的预处理方法（如缩放或裁剪）可能造成大量细节损失。为解决这一问题，我们设计了一个多分支深度神经网络（DNN），从三个视角评估UHD图像的质量：全局美学特征、局部技术失真和显著内容感知。具体而言，美学特征从UHD图像下采样得到的低分辨率图像中提取，这些低分辨率图像丢失了高频纹理信息，但仍保留全局美学特征。技术失真通过一个由基于网格迷你块采样策略从UHD图像中裁剪的迷你块组成的碎片图像来测量。UHD图像的显著内容被检测并裁剪，以从显著区域提取质量感知特征。我们采用Swin Transformer Tiny作为骨干网络，从这三个视角提取特征。提取的特征被拼接后，通过一个两层多层感知机（MLP）网络回归为质量分数。我们采用均方误差（MSE）损失来优化预测精度，并采用保真度损失来优化预测单调性。实验结果表明，所提出的模型在UHD-IQA数据集上取得了最佳性能，同时保持了最低的计算复杂度，证明了其有效性和高效性。此外，所提出的模型在ECCV AIM 2024 UHD-IQA挑战赛中获得第一名。代码可在https://github.com/sunwei925/UIQA获取。
英文摘要：
UHD images, typically with resolutions equal to or higher than 4K, pose a significant challenge for efficient image quality assessment (IQA) algorithms, as adopting full-resolution images as inputs leads to overwhelming computational complexity and commonly used pre-processing methods like resizing or cropping may cause substantial loss of detail. To address this problem, we design a multi-branch deep neural network (DNN) to assess the quality of UHD images from three perspectives: global aesthetic characteristics, local technical distortions, and salient content perception. Specifically, aesthetic features are extracted from low-resolution images downsampled from the UHD ones, which lose high-frequency texture information but still preserve the global aesthetics characteristics. Technical distortions are measured using a fragment image composed of mini-patches cropped from UHD images based on the grid mini-patch sampling strategy. The salient content of UHD images is detected and cropped to extract quality-aware features from the salient regions. We adopt the Swin Transformer Tiny as the backbone networks to extract features from these three perspectives. The extracted features are concatenated and regressed into quality scores by a two-layer multi-layer perceptron (MLP) network. We employ the mean square error (MSE) loss to optimize prediction accuracy and the fidelity loss to optimize prediction monotonicity. Experimental results show that the proposed model achieves the best performance on the UHD-IQA dataset while maintaining the lowest computational complexity, demonstrating its effectiveness and efficiency. Moreover, the proposed model won first prize in ECCV AIM 2024 UHD-IQA Challenge. The code is available at https://github.com/sunwei925/UIQA.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
