基于生成式扫描路径表示的360°图像感知质量评估
Perceptual Quality Assessment of 360$^\circ$ Images Based on Generative Scanpath Representation
AI总结：
针对360°图像质量评估忽视观看行为多样性的问题，提出生成式扫描路径表示（GSR）与计算框架，通过聚合多假设用户的动态视觉注视，有效提升了不同观看条件下局部失真图像质量预测与人类感知的一致性。
摘要：
尽管在启发式模型设计上为全方位（即360°）图像质量评估（OIQA）付出了大量努力，但由于缺乏对导致360°图像感知质量变化的观看行为多样性的考虑，仍然存在明显的差距。两个关键方面凸显了这一疏忽：忽视了显著影响用户注视模式的观看条件，以及过度依赖来自360°图像的单一视口序列进行质量推断。为了解决这些问题，我们引入了一种独特的生成式扫描路径表示（GSR），用于360°图像的有效质量推断，它聚合了在预定义观看条件下多假设用户的不同感知体验。更具体地说，给定由观看起点和探索时间表征的观看条件，可以使用合适的扫描路径生成器生成一组包含动态视觉注视的扫描路径。沿着这一思路，我们使用扫描路径将360°图像转换为独特的GSR，它提供了源自扫描路径的注视焦点内容的全局概览。因此，360°图像的质量推断被迅速转化为GSR的质量推断。然后，我们通过学习GSR的质量图，提出了一个高效的OIQA计算框架。综合实验结果验证了所提出框架的预测在时空域与人类感知高度一致，特别是在不同观看条件下局部失真的360°图像这一具有挑战性的环境中。代码将发布于 https://github.com/xiangjieSui/GSR
英文摘要：
Despite substantial efforts dedicated to the design of heuristic models for omnidirectional (i.e., 360$^\circ$) image quality assessment (OIQA), a conspicuous gap remains due to the lack of consideration for the diversity of viewing behaviors that leads to the varying perceptual quality of 360$^\circ$ images. Two critical aspects underline this oversight: the neglect of viewing conditions that significantly sway user gaze patterns and the overreliance on a single viewport sequence from the 360$^\circ$ image for quality inference. To address these issues, we introduce a unique generative scanpath representation (GSR) for effective quality inference of 360$^\circ$ images, which aggregates varied perceptual experiences of multi-hypothesis users under a predefined viewing condition. More specifically, given a viewing condition characterized by the starting point of viewing and exploration time, a set of scanpaths consisting of dynamic visual fixations can be produced using an apt scanpath generator. Following this vein, we use the scanpaths to convert the 360$^\circ$ image into the unique GSR, which provides a global overview of gazed-focused contents derived from scanpaths. As such, the quality inference of the 360$^\circ$ image is swiftly transformed to that of GSR. We then propose an efficient OIQA computational framework by learning the quality maps of GSR. Comprehensive experimental results validate that the predictions of the proposed framework are highly consistent with human perception in the spatiotemporal domain, especially in the challenging context of locally distorted 360$^\circ$ images under varied viewing conditions. The code will be released at https://github.com/xiangjieSui/GSR
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
