arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2411.18372cs.MM
收藏

不确定性驱动的采样方法用于高效成对比较主观评估

Uncertainty-driven Sampling for Efficient Pairwise Comparison Subjective Assessment

Shima Mohammadi, João Ascenso

AI总结:

本文提出一种基于不确定性的采样方法,利用深度学习模型估计人类偏好并识别需人工标注的图像对,以减少成对比较主观评估中的比较次数,同时保持高准确性,实验证明优于传统主动采样方法。

摘要:

在图像压缩、超分辨率、去噪等图像处理任务中,评估图像质量至关重要。尽管涉及人类评估者的主观评估能提供最准确的质量分数,但由于其高昂的成本和时间需求,在大规模或持续评估中并不实用。成对比较主观评估测试通过对图像对进行排序而非打分,提供了更高的可靠性和准确性,但需要大量的比较,导致成本高昂。虽然客观质量指标更为高效,但它们缺乏主观测试的精确性,而主观测试对于基准测试和训练基于学习的质量指标至关重要。本文提出了一种基于不确定性的采样方法,以优化成对比较主观评估过程。通过利用深度学习模型来估计人类偏好并识别需要人工标注的图像对,该方法在保持高准确性的同时减少了所需比较的数量。主要贡献包括对不确定性进行建模,以实现准确的偏好预测和成对采样。实验结果表明,与传统主动采样方法相比,所提出的方法具有优越的性能。软件公开可用:shimamohammadi/LBPS-EIC

英文摘要:

Assessing image quality is crucial in image processing tasks such as compression, super-resolution, and denoising. While subjective assessments involving human evaluators provide the most accurate quality scores, they are impractical for large-scale or continuous evaluations due to their high cost and time requirements. Pairwise comparison subjective assessment tests, which rank image pairs instead of assigning scores, offer more reliability and accuracy but require numerous comparisons, leading to high costs. Although objective quality metrics are more efficient, they lack the precision of subjective tests, which are essential for benchmarking and training learning-based quality metrics. This paper proposes an uncertainty-based sampling method to optimize the pairwise comparison subjective assessment process. By utilizing deep learning models to estimate human preferences and identify pairs that need human labeling, the approach reduces the number of required comparisons while maintaining high accuracy. The key contributions include modeling uncertainty for accurate preference predictions and for pairwise sampling. The experimental results demonstrate superior performance of the proposed approach compared to traditional active sampling methods. Software is publicly available at: shimamohammadi/LBPS-EIC

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑