arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2410.11528cs.CV
收藏

Hairmony:公平性感知的发型分类

Hairmony: Fairness-aware hairstyle classification

Microsoft(微软);The Pennsylvania State University(宾夕法尼亚州立大学)
Givi Meishvili, James Clemoes, Charlie Hewitt, Zafiirah Hosenie, Xian Xiao, Martin de La Gorce, Tibor Takacs, Tadas Baltrusaitis, Antonio Criminisi, Chyna McRae… 展开作者

Givi Meishvili, James Clemoes, Charlie Hewitt, Zafiirah Hosenie, Xian Xiao, Martin de La Gorce, Tibor Takacs, Tadas Baltrusaitis, Antonio Criminisi, Chyna McRae, Nina Jablonski, Marta Wilczkowiak

AI总结:

提出基于合成数据和新型发型分类法的公平性感知发型分类方法,采用预训练特征提取网络架构,在挑战性发型上显著优于参数化方法。

摘要:

我们提出了一种从单张图像预测人物发型的方法。尽管在虚拟体验的用户数字化和注册中应用日益增多,但现有方法仍然有限,特别是在它们能捕捉的发型范围上。人类头发极其多样,且缺乏任何普遍接受的描述或分类,这使得该任务极具挑战性。当前大多数方法依赖于发丝级别的参数化头发模型。这些方法虽然很有前景,但尚无法表现短发、卷曲发、螺旋卷发和束发。我们转而选择一种分类方法,能够表示真正稳健和包容性系统所需的发型多样性。以往的分类方法受限于缺乏多样性的标注不良数据,这制约了任何最终注册系统的实用性。我们仅使用合成数据来训练模型。这允许对发型属性、发色、面部外观、姿态、环境和其他参数的多样性进行显式控制,同时也产生无噪声的真实标签。我们引入了一种与多元化领域专家团队合作开发的新型发型分类法,用于平衡训练数据、监督模型并直接衡量公平性。我们使用该分类法标注了合成训练数据和真实评估数据集,并发布两者以支持未来发型预测方法的比较。我们采用基于预训练特征提取网络的架构,以提高方法对真实数据的泛化能力,并将预测分类法属性作为辅助任务来提升准确性。结果表明,与近期的参数化方法相比,我们的方法在处理具有挑战性的发型时显著更稳健。

英文摘要:

We present a method for prediction of a person's hairstyle from a single image. Despite growing use cases in user digitization and enrollment for virtual experiences, available methods are limited, particularly in the range of hairstyles they can capture. Human hair is extremely diverse and lacks any universally accepted description or categorization, making this a challenging task. Most current methods rely on parametric models of hair at a strand level. These approaches, while very promising, are not yet able to represent short, frizzy, coily hair and gathered hairstyles. We instead choose a classification approach which can represent the diversity of hairstyles required for a truly robust and inclusive system. Previous classification approaches have been restricted by poorly labeled data that lacks diversity, imposing constraints on the usefulness of any resulting enrollment system. We use only synthetic data to train our models. This allows for explicit control of diversity of hairstyle attributes, hair colors, facial appearance, poses, environments and other parameters. It also produces noise-free ground-truth labels. We introduce a novel hairstyle taxonomy developed in collaboration with a diverse group of domain experts which we use to balance our training data, supervise our model, and directly measure fairness. We annotate our synthetic training data and a real evaluation dataset using this taxonomy and release both to enable comparison of future hairstyle prediction approaches. We employ an architecture based on a pre-trained feature extraction network in order to improve generalization of our method to real data and predict taxonomy attributes as an auxiliary task to improve accuracy. Results show our method to be significantly more robust for challenging hairstyles than recent parametric approaches.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑