arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2402.17113cs.CVcs.GR
收藏

基于潜在透明度的透明图像层扩散

Transparent Image Layer Diffusion using Latent Transparency

Stanford University(斯坦福大学)

Lvmin Zhang, Maneesh Agrawala

AI总结:

该研究提出LayerDiffuse方法,通过学习潜在透明度微调预训练潜在扩散模型以生成透明图像,支持多类应用,用户偏好度和素材质量均表现优异。

摘要:

我们提出了LayerDiffuse,一种使大规模预训练潜在扩散模型能够生成透明图像的方法。该方法支持生成单张透明图像或多个透明图层。该方法学习一种“潜在透明度”,将alpha通道透明度编码到预训练潜在扩散模型的潜在流形中。它通过将新增的透明度作为潜在偏移进行调节,仅对预训练模型的原始潜在分布做最小改动,从而保留了大型扩散模型的生产级质量。通过这种方式,任何潜在扩散模型都可以通过在调整后的潜在空间中进行微调,被转换为透明图像生成器。我们采用人在环路采集方案收集了100万对透明图像层对,并用其训练模型。我们表明,潜在透明度可以应用于不同的开源图像生成器,也可以适配各种条件控制系统,以实现前景/背景条件图层生成、联合图层生成、图层内容结构控制等应用。一项用户研究发现,在大多数情况下(97%),用户更偏好我们原生生成的透明内容,而非以往的临时解决方案(如先生成图像再做抠图)。用户还表示,我们生成的透明图像质量可与Adobe Stock等真实商业透明素材相媲美。

英文摘要:

We present LayerDiffuse, an approach enabling large-scale pretrained latent diffusion models to generate transparent images. The method allows generation of single transparent images or of multiple transparent layers. The method learns a "latent transparency" that encodes alpha channel transparency into the latent manifold of a pretrained latent diffusion model. It preserves the production-ready quality of the large diffusion model by regulating the added transparency as a latent offset with minimal changes to the original latent distribution of the pretrained model. In this way, any latent diffusion model can be converted into a transparent image generator by finetuning it with the adjusted latent space. We train the model with 1M transparent image layer pairs collected using a human-in-the-loop collection scheme. We show that latent transparency can be applied to different open source image generators, or be adapted to various conditional control systems to achieve applications like foreground/background-conditioned layer generation, joint layer generation, structural control of layer contents, etc. A user study finds that in most cases (97%) users prefer our natively generated transparent content over previous ad-hoc solutions such as generating and then matting. Users also report the quality of our generated transparent images is comparable to real commercial transparent assets like Adobe Stock.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑