Pose-Star:面向开放世界时尚图像的解剖感知编辑
Pose-Star: Anatomy-Aware Editing for Open-World Fashion Images
AI总结:
Pose-Star提出解剖感知掩码框架,通过骨骼关键点校准注意力并抑制噪声,解决时尚图像编辑中掩码可控性差和姿态鲁棒性弱的问题。
摘要:
为了推进真实世界时尚图像编辑,我们分析了现有的两阶段流程(掩码生成后接基于扩散的编辑),这些流程过度优先考虑生成器优化,而忽视了掩码可控性。这导致两个关键局限:I)用户定义的灵活性差(粗粒度人体掩码将编辑限制在预定义区域,如上半身;细粒度服装掩码保留姿态,但禁止款式/长度定制)。II)姿态鲁棒性弱(掩码生成器因关节姿态而失败,并遗漏腰部等稀有区域,而人体解析器仍受预定义类别限制)。为解决这些不足,我们提出Pose-Star,一个将身体结构(如颈部、胸部等)动态重组为解剖感知掩码(如齐胸长度)以进行用户定义编辑的框架。在Pose-Star中,我们通过骨骼关键点校准扩散派生的注意力(Star tokens),以增强复杂姿态下稀有结构的定位,通过注意力动态的相位感知分析(收敛、稳定、发散)结合阈值掩码和滑动窗口融合抑制噪声,并通过跨自注意力合并和Canny对齐细化边缘。这项工作连接了受控基准与开放世界需求,开创了解剖感知、姿态鲁棒的编辑,并为工业时尚图像编辑奠定基础。
英文摘要:
To advance real-world fashion image editing, we analyze existing two-stage pipelines(mask generation followed by diffusion-based editing)which overly prioritize generator optimization while neglecting mask controllability. This results in two critical limitations: I) poor user-defined flexibility (coarse-grained human masks restrict edits to predefined regions like upper torso; fine-grained clothes masks preserve poses but forbid style/length customization). II) weak pose robustness (mask generators fail due to articulated poses and miss rare regions like waist, while human parsers remain limited by predefined categories). To address these gaps, we propose Pose-Star, a framework that dynamically recomposes body structures (e.g., neck, chest, etc.) into anatomy-aware masks (e.g., chest-length) for user-defined edits. In Pose-Star, we calibrate diffusion-derived attention (Star tokens) via skeletal keypoints to enhance rare structure localization in complex poses, suppress noise through phase-aware analysis of attention dynamics (Convergence,Stabilization,Divergence) with threshold masking and sliding-window fusion, and refine edges via cross-self attention merging and Canny alignment. This work bridges controlled benchmarks and open-world demands, pioneering anatomy-aware, pose-robust editing and laying the foundation for industrial fashion image editing.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。