arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2303.02700cs.CV
收藏

HairStep：使用单视图3D头发建模的股和深度贴图将合成转换为真实

HairStep: Transfer Synthetic to Real Using Strand and Depth Maps for Single-View 3D Hair Modeling

CUHKSZ（香港中文大学（深圳））；Kuaishou Technology（快手科技）

Yujian Zheng, Zirong Jin, Moran Li, Haibin Huang, Chongyang Ma, Shuguang Cui, Xiaoguang Han

阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

摘要：

在这项工作中，我们解决了基于学习的单视图3D头发建模的挑战性问题。由于采集成对的真实图像和三维毛发数据的难度很大，利用合成数据为真实域提供先验知识成为一种主流解决方案。不幸的是，这引入了域差距的挑战。由于真实感头发渲染的固有困难，现有方法通常使用方向图而不是头发图像作为输入来弥合差距。我们坚定地认为一个中间表示是必不可少的，但我们认为，方向图使用占主导地位的基于过滤的方法是敏感的不确定的噪音，远远不能胜任的表示。因此，我们首先提出这个问题，并提出了一种新的中间表示，称为HairStep，它由一个链图和一个深度图。结果表明，HairStep不仅为精确的三维头发建模提供了足够的信息，而且可以从真实图像中推断出头发的形状。具体来说，我们收集了一个包含1,250张肖像图像的数据集，其中包含两种类型的注释。进一步设计了一个学习框架，将真实图像转换为链图和深度图。值得注意的是，我们的新数据集的额外好处是3D头发建模的第一个定量指标。我们的实验表明，HairStep缩小了合成和真实之间的域差距，并在单视图3D头发重建上实现了最先进的性能。

英文摘要：

In this work, we tackle the challenging problem of learning-based single-view 3D hair modeling. Due to the great difficulty of collecting paired real image and 3D hair data, using synthetic data to provide prior knowledge for real domain becomes a leading solution. This unfortunately introduces the challenge of domain gap. Due to the inherent difficulty of realistic hair rendering, existing methods typically use orientation maps instead of hair images as input to bridge the gap. We firmly think an intermediate representation is essential, but we argue that orientation map using the dominant filtering-based methods is sensitive to uncertain noise and far from a competent representation. Thus, we first raise this issue up and propose a novel intermediate representation, termed as HairStep, which consists of a strand map and a depth map. It is found that HairStep not only provides sufficient information for accurate 3D hair modeling, but also is feasible to be inferred from real images. Specifically, we collect a dataset of 1,250 portrait images with two types of annotations. A learning framework is further designed to transfer real images to the strand map and depth map. It is noted that, an extra bonus of our new dataset is the first quantitative metric for 3D hair modeling. Our experiments show that HairStep narrows the domain gap between synthetic and real and achieves state-of-the-art performance on single-view 3D hair reconstruction.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑