使用上下文敏感模型集成的图像相似性
Image Similarity using An Ensemble of Context-Sensitive Models
AI总结：
针对图像相似性数值标注困难的问题，提出基于 A:R 对 B:R 相对比较标注的上下文敏感模型集成方法，性能优于最佳单模型约 5%，并超过混合数据微调模型及 CLIP、DINO 深度嵌入。
摘要：
图像相似性在计算机视觉中已得到广泛研究。近年来，机器学习模型已显示出比传统多变量度量编码更多语义的能力。然而，在标注语义相似性时，为一对图像分配数值分数并不现实，这使得该任务的改进和比较变得困难。在这项工作中，我们提出了一种更直观的方法，基于 A:R 与 B:R 形式的标注数据来构建和比较图像相似性模型，即判断图像 A 是否比另一图像 B 更接近参考图像 R。我们通过使用集成模型，解决了图像空间 (R, A, B) 中稀疏采样的挑战，以及使用基于上下文的数据训练的模型中的偏差。我们的测试结果表明，所构建的集成模型比最佳的单个上下文敏感模型性能提高约 5%。它们的表现也优于直接使用混合图像数据微调的模型以及现有深度嵌入，例如 CLIP 和 DINO。这项工作表明，当使用适当的集成方法来缓解稀疏采样导致的限制时，基于上下文的标注和模型训练可以是有效的。
英文摘要：
Image similarity has been extensively studied in computer vision. In recent years, machine-learned models have shown their ability to encode more semantics than traditional multivariate metrics. However, in labelling semantic similarity, assigning a numerical score to a pair of images is impractical, making the improvement and comparisons on the task difficult. In this work, we present a more intuitive approach to build and compare image similarity models based on labelled data in the form of A:R vs B:R, i.e., determining if an image A is closer to a reference image R than another image B. We address the challenges of sparse sampling in the image space (R, A, B) and biases in the models trained with context-based data by using an ensemble model. Our testing results show that the ensemble model constructed performs ~5% better than the best individual context-sensitive models. They also performed better than the models that were directly fine-tuned using mixed imagery data as well as existing deep embeddings, e.g., CLIP and DINO. This work demonstrates that context-based labelling and model training can be effective when an appropriate ensemble approach is used to alleviate the limitation due to sparse sampling.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
