arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2305.15367cs.CVcs.AI
收藏

SAMScore:一种用于图像翻译评估的内容结构相似性度量

SAMScore: A Content Structural Similarity Metric for Image Translation Evaluation

The University of Texas Southwestern Medical Center(德克萨斯大学西南医学中心);University of Toronto(多伦多大学);The University of Texas at Austin(德克萨斯大学奥斯汀分校)

Yunxiang Li, Meixu Chen, Kai Wang, Jun Ma, Alan C. Bovik, You Zhang

AI总结:

针对图像翻译中内容结构保真度难以评估的问题,提出基于Segment Anything Model的内容结构相似性度量SAMScore,在19个任务上优于其他竞争度量,助力图像翻译模型的精确评估。

摘要:

图像翻译具有广泛的应用,例如风格迁移和模态转换,通常旨在生成兼具高度真实感和保真度的图像。这些问题仍然具有挑战性,特别是在需要保留内容结构时尤为重要。传统的图像级相似性度量作用有限,因为图像的内容结构属于高层信息,并不强烈受制于与原图的像素级保真度。为了填补这一空白,我们引入了SAMScore,这是一种通用的内容结构相似性度量,用于评估图像翻译模型的保真度。SAMScore基于近期高性能的Segment Anything Model (SAM),能够以突出的准确度进行内容相似性比较。我们在19个图像翻译任务上应用了SAMScore,发现它能够在所有任务上优于所有其他竞争性度量。我们设想SAMScore将成为一种有价值的工具,通过允许对新兴和不断演进的翻译模型进行更精确的评估,推动图像翻译这一活跃领域的发展。代码可在https://github.com/Kent0n-Li/SAMScore获取。

英文摘要:

Image translation has wide applications, such as style transfer and modality conversion, usually aiming to generate images having both high degrees of realism and faithfulness. These problems remain difficult, especially when it is important to preserve content structures. Traditional image-level similarity metrics are of limited use, since the content structures of an image are high-level, and not strongly governed by pixel-wise faithfulness to an original image. To fill this gap, we introduce SAMScore, a generic content structural similarity metric for evaluating the faithfulness of image translation models. SAMScore is based on the recent high-performance Segment Anything Model (SAM), which allows content similarity comparisons with standout accuracy. We applied SAMScore on 19 image translation tasks, and found that it is able to outperform all other competitive metrics on all tasks. We envision that SAMScore will prove to be a valuable tool that will help to drive the vibrant field of image translation, by allowing for more precise evaluations of new and evolving translation models. The code is available at https://github.com/Kent0n-Li/SAMScore.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑