CoIE：面向多属性人脸编辑的指令链编辑方法
CoIE: Chain-of-Instruct Editing for Multi-Attribute Face Manipulation
AI总结：
针对单指令多属性人脸编辑难题，提出CoIE指令链编辑方法，结合大模型生成分步指令、自建数据集微调及超分辨率模块，显著提升编辑成功率与可控性。
摘要：
当前的文本到图像编辑模型在通过单条指令平滑操控多个属性时往往面临挑战。受大语言模型中思维链提示技术的启发，我们提出了名为Chain-of-Instruct Editing（指令链编辑，简称CoIE）的创新思路，通过一系列指令分步编辑来提升这类模型的能力。具体而言，在人脸编辑场景下，我们利用GPT-4等预训练大语言模型（LLM）的上下文学习能力，通过专门设计的1-shot（单样本）模板从原始输入中生成指令序列。为进一步提升每一步编辑的精度，我们使用自建的指令引导人脸编辑数据集Instruct-CelebA对编辑模型进行微调。此外，我们还加入了超分辨率模块，以缓解编辑能力下降和画质退化的负面影响。在多种具有挑战性的案例上的实验结果证实，指令链编辑显著提升了多属性人脸图像编辑效果：以CLIPSim和Coverage指标衡量的编辑成功率分别提升了17.86%和85.45%，以Preserve L1和Quality指标体现的可控性分别提升了11.58%和4.93%。
英文摘要：
Current text-to-image editing models often encounter challenges with smoothly manipulating multiple attributes using a single instruction. Taking inspiration from the Chain-of-Thought prompting technique utilized in language models, we present an innovative concept known as Chain-of-Instruct Editing (CoIE), which enhances the capabilities of these models through step-by-step editing using a series of instructions. In particular, in the context of face manipulation, we leverage the contextual learning abilities of a pretrained Large Language Model (LLM), such as GPT-4, to generate a sequence of instructions from the original input, utilizing a purpose-designed 1-shot template. To further improve the precision of each editing step, we conduct fine-tuning on the editing models using our self-constructed instruction-guided face editing dataset, Instruct-CelebA. And additionally, we incorporate a super-resolution module to mitigate the adverse effects of editability and quality degradation. Experimental results across various challenging cases confirm the significant boost in multi-attribute facial image manipulation using chain-of-instruct editing. This is evident in enhanced editing success rates, measured by CLIPSim and Coverage metrics, improved by 17.86% and 85.45% respectively, and heightened controllability indicated by Preserve L1 and Quality metrics, improved by 11.58% and 4.93% respectively.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
