MagiCapture：高分辨率多概念人像定制
MagiCapture: High-Resolution Multi-Concept Portrait Customization
AI总结：
针对现有文本生成图像个性化方法人像真实度不足、易出现身份偏移的问题，提出MagiCapture方法，结合注意力重聚焦损失与辅助先验，仅需少量参考即可生成高分辨率多概念人像，效果优于基准且可推广到非人体物体。
摘要：
包括Stable Diffusion在内的大规模文本生成图像模型能够生成高保真的写实人像图像。目前有一个活跃的研究领域致力于这类模型的个性化，旨在利用提供的参考图像集合合成特定的主体或风格。然而，尽管这些个性化方法能生成看似合理的结果，但它们产出的图像往往真实度不足，尚未达到商业可行的水平。这一点在人像生成中尤为明显，由于人类固有的感知偏差，人脸上任何不自然的伪影都很容易被察觉。为解决这一问题，我们提出了MagiCapture，这是一种整合主体与风格概念的个性化方法，仅需少量主体和风格参考图像就能生成高分辨率人像。例如，给定几张随机的自拍，我们经过微调的模型就能生成特定风格的高质量人像，比如护照照或个人资料照。该任务的主要挑战在于组合后的概念不存在真实标注（ground truth），这会导致最终输出质量下降，且源主体的身份发生偏移。为解决这些问题，我们提出了一种新颖的注意力重聚焦（Attention Refocusing）损失函数，搭配辅助先验，二者共同助力在这种弱监督学习设置下实现鲁棒学习。我们的流程还包含额外的后处理步骤，以确保生成高度真实的输出。MagiCapture在定量和定性评估中均优于其他基准方法，还能推广到其他非人类物体。
英文摘要：
Large-scale text-to-image models including Stable Diffusion are capable of generating high-fidelity photorealistic portrait images. There is an active research area dedicated to personalizing these models, aiming to synthesize specific subjects or styles using provided sets of reference images. However, despite the plausible results from these personalization methods, they tend to produce images that often fall short of realism and are not yet on a commercially viable level. This is particularly noticeable in portrait image generation, where any unnatural artifact in human faces is easily discernible due to our inherent human bias. To address this, we introduce MagiCapture, a personalization method for integrating subject and style concepts to generate high-resolution portrait images using just a few subject and style references. For instance, given a handful of random selfies, our fine-tuned model can generate high-quality portrait images in specific styles, such as passport or profile photos. The main challenge with this task is the absence of ground truth for the composed concepts, leading to a reduction in the quality of the final output and an identity shift of the source subject. To address these issues, we present a novel Attention Refocusing loss coupled with auxiliary priors, both of which facilitate robust learning within this weakly supervised learning setting. Our pipeline also includes additional post-processing steps to ensure the creation of highly realistic outputs. MagiCapture outperforms other baselines in both quantitative and qualitative evaluations and can also be generalized to other non-human objects.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
