重新构建图像差异描述：基于BLIP2IDC与合成增强
Reframing Image Difference Captioning with BLIP2IDC and Synthetic Augmentation
AI总结：
本文提出BLIP2IDC框架，将BLIP2适配到图像差异描述任务，并利用合成增强生成高质量数据集Syned1，显著提升真实图像上的差异描述性能。
摘要：
近年来，生成模型质量的提升使得大规模生成图像的编辑变体成为可能。为了应对此类技术可能带来的有害影响，图像差异描述（IDC）任务旨在描述两幅图像之间的差异。尽管该任务在简单的3D渲染图像上已得到成功处理，但在真实世界图像上仍面临挑战。其原因有二：训练数据稀缺，以及难以捕捉复杂图像之间的细微差异。为解决这些问题，本文提出了一种简单而有效的框架，既能将现有图像描述模型适配到IDC任务，又能增强IDC数据集。我们引入了BLIP2IDC，这是BLIP2在低计算成本下对IDC任务的适配，并展示了其在真实世界IDC数据集上显著优于双流方法。我们还提出使用合成增强来以与模型无关的方式提升IDC模型的性能。我们证明，我们的合成增强策略提供了高质量的数据，从而产生了一个具有挑战性的新数据集Syned1，非常适合IDC任务。
英文摘要：
The rise of the generative models quality during the past years enabled the generation of edited variations of images at an important scale. To counter the harmful effects of such technology, the Image Difference Captioning (IDC) task aims to describe the differences between two images. While this task is successfully handled for simple 3D rendered images, it struggles on real-world images. The reason is twofold: the training data-scarcity, and the difficulty to capture fine-grained differences between complex images. To address those issues, we propose in this paper a simple yet effective framework to both adapt existing image captioning models to the IDC task and augment IDC datasets. We introduce BLIP2IDC, an adaptation of BLIP2 to the IDC task at low computational cost, and show it outperforms two-streams approaches by a significant margin on real-world IDC datasets. We also propose to use synthetic augmentation to improve the performance of IDC models in an agnostic fashion. We show that our synthetic augmentation strategy provides high quality data, leading to a challenging new dataset well-suited for IDC named Syned1.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
