SSCR：通过自监督反事实推理实现基于语言的迭代图像编辑
SSCR: Iterative Language-Based Image Editing via Self-Supervised Counterfactual Reasoning
AI总结：
针对迭代语言图像编辑的数据稀缺问题，提出自监督反事实推理框架 SSCR，利用分布外指令和 CTC 自监督训练，在 i-CLEVR 与 CoDraw 上达到 SOTA，且半量数据即可取得接近全量结果。
摘要：
基于语言的迭代图像编辑（ILBIE）任务遵循迭代指令逐步编辑图像。数据稀缺是 ILBIE 的一个重要问题，因为收集指令引发变化前后的大规模图像示例颇具挑战。然而，人类即使面对不熟悉的图像—指令配对，仍能完成这些编辑任务。这种能力源于反事实思维，即思考已发生事件的替代情况的能力。本文提出一种自监督反事实推理（Self-Supervised Counterfactual Reasoning，SSCR）框架，将反事实思维纳入其中以克服数据稀缺。SSCR 使模型能够考虑与先前图像配对的分布外指令。在跨任务一致性（cross-task consistency，CTC）的帮助下，我们在自监督场景中训练这些反事实指令。大量结果表明，SSCR 在对象身份和位置两方面提升了 ILBIE 的正确性，在两个 IBLIE 数据集（i-CLEVR 和 CoDraw）上建立了新的最优水平（SOTA）。即使仅使用 50% 的训练数据，SSCR 也能取得与使用完整数据相当的结果。
英文摘要：
Iterative Language-Based Image Editing (IL-BIE) tasks follow iterative instructions to edit images step by step. Data scarcity is a significant issue for ILBIE as it is challenging to collect large-scale examples of images before and after instruction-based changes. However, humans still accomplish these editing tasks even when presented with an unfamiliar image-instruction pair. Such ability results from counterfactual thinking and the ability to think about alternatives to events that have happened already. In this paper, we introduce a Self-Supervised Counterfactual Reasoning (SSCR) framework that incorporates counterfactual thinking to overcome data scarcity. SSCR allows the model to consider out-of-distribution instructions paired with previous images. With the help of cross-task consistency (CTC), we train these counterfactual instructions in a self-supervised scenario. Extensive results show that SSCR improves the correctness of ILBIE in terms of both object identity and position, establishing a new state of the art (SOTA) on two IBLIE datasets (i-CLEVR and CoDraw). Even with only 50% of the training data, SSCR achieves a comparable result to using complete data.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
