带空间约束的文本引导眼镜操控
Text-guided Eyeglasses Manipulation with Spatial Constraints
AI总结：
针对现有眼镜虚拟试穿方法风格有限、交互不直观的问题，提出文本引导的眼镜操控方法，结合掩码空间约束与文本条件，通过两阶段训练实现眼镜形状风格的细粒度控制，提升局部编辑效果。
摘要：
眼镜虚拟试穿旨在无需实际佩戴即可将不同形状和款式的眼镜合成到人脸图像上。现有方法虽已取得不错效果，但眼镜风格种类有限，交互方式也往往不够直观高效。针对这些局限，我们提出一种文本引导的眼镜操控方法，可分别基于二值掩码和文本控制眼镜的形状与风格。具体而言，我们引入掩码编码器提取掩码条件，并设计调制模块实现文本条件与掩码条件的同时注入，该设计能结合文本描述和空间约束对眼镜外观进行细粒度控制。方法还包含解耦映射器和去耦策略以保留无关区域，实现更优的局部编辑效果。我们采用两阶段训练方案应对不同模态条件的收敛速度差异，成功实现眼镜形状和风格的双重控制。大量对比实验和消融分析表明，该方法既能生成多样的眼镜风格，又能有效保留无关区域。
英文摘要：
Virtual try-on of eyeglasses involves placing eyeglasses of different shapes and styles onto a face image without physically trying them on. While existing methods have shown impressive results, the variety of eyeglasses styles is limited and the interactions are not always intuitive or efficient. To address these limitations, we propose a Text-guided Eyeglasses Manipulation method that allows for control of the eyeglasses shape and style based on a binary mask and text, respectively. Specifically, we introduce a mask encoder to extract mask conditions and a modulation module that enables simultaneous injection of text and mask conditions. This design allows for fine-grained control of the eyeglasses' appearance based on both textual descriptions and spatial constraints. Our approach includes a disentangled mapper and a decoupling strategy that preserves irrelevant areas, resulting in better local editing. We employ a two-stage training scheme to handle the different convergence speeds of the various modality conditions, successfully controlling both the shape and style of eyeglasses. Extensive comparison experiments and ablation analyses demonstrate the effectiveness of our approach in achieving diverse eyeglasses styles while preserving irrelevant areas.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
