Ar2Can:利用画布进行多人体生成的架构与艺术家
Ar2Can: An Architect and an Artist Leveraging a Canvas for Multi-Human Generation
AI总结:
Ar2Can提出一种两阶段框架,通过分离空间规划与身份渲染,解决多人体生成中人脸身份丢失问题,采用空间引导的面部匹配奖励提升生成质量,使用合成数据实现高精度和保真度。
摘要:
尽管个性化图像生成取得进展,现有模型在生成多人体场景时仍常导致人脸身份混淆。我们提出Ar2Can,一种两阶段框架,将空间规划与身份渲染解耦。架构预测结构布局,指定每个人出现位置;艺术家则通过结合匈牙利空间对齐与身份相似性的空间引导面部匹配奖励生成逼真图像。该方法确保人脸正确位置并忠实保留参考身份。我们开发了两种架构变体,与基于扩散的艺术家模型无缝集成。通过组相对策略优化(GRPO)使用组合奖励优化计数精度、图像质量和身份匹配。在MultiHuman-Testbench上评估,Ar2Can在计数精度和身份保真度上取得显著提升,同时保持高质量。值得注意的是,我们的方法主要使用合成数据,无需真实多人体图像。项目页面:https://qualcomm-ai-research.github.io/ar2can/.
英文摘要:
Despite recent advances in personalized image generation, existing models consistently fail to produce reliable multi-human scenes, often merging or losing facial identity. We present Ar2Can, a novel two-stage framework that disentangles spatial planning from identity rendering for multi-human generation. The Architect predicts structured layouts, specifying where each person should appear. The Artist then synthesizes photorealistic images, guided by a spatially-grounded face matching reward that combines Hungarian spatial alignment with identity similarity. This approach ensures faces are rendered at correct locations and faithfully preserve reference identities. We develop two Architect variants, seamlessly integrated with our diffusion-based Artist model. This is optimized via Group Relative Policy Optimization (GRPO) using compositional rewards for count accuracy, image quality, and identity matching. Evaluated on the MultiHuman-Testbench, Ar2Can achieves substantial improvements in both count accuracy and identity preservation, while maintaining high perceptual quality. Notably, our method achieves these results using primarily synthetic data, without requiring real multi-human images. Project page: https://qualcomm-ai-research.github.io/ar2can/.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。