arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2412.20901cs.CVcs.AI
收藏

ILDiff:通过隐式布局蒸馏生成透明动画贴纸

ILDiff: Generate Transparent Animated Stickers by Implicit Layout Distillation

Wechat AI, Tencent(腾讯微信人工智能)

Ting Zhang, Zhiqiang Yuan, Yeshuang Zhu, Jinchao Zhang

AI总结:

本文提出ILDiff方法,通过隐式布局蒸馏生成动画透明通道,解决半开放区域塌陷与时间信息缺失问题,并构建含0.32M样本的TASD数据集,实验证明其透明通道更精细平滑。

摘要:

高质量动画贴纸通常包含透明通道,而当前视频生成模型往往忽略这一通道。为了生成细粒度的动画透明通道,现有方法大致可分为视频抠图算法和基于扩散的算法。基于视频抠图的方法在处理贴纸中的半开放区域时表现较差,而基于扩散的方法通常用于建模单张图像,这会在建模动画贴纸时导致局部闪烁。在本文中,我们首先提出了一种ILDiff方法,通过隐式布局蒸馏生成动画透明通道,解决了现有方法中半开放区域塌陷以及未考虑时间信息的问题。其次,我们创建了透明动画贴纸数据集(TASD),该数据集包含0.32M个带有透明通道的高质量样本,为相关领域提供数据支持。大量实验表明,与Matting Anything和Layer Diffusion等其他方法相比,ILDiff能够生成更精细、更平滑的透明通道。我们的代码和数据集将在链接https://xiaoyuan1996.github.io发布。

英文摘要:

High-quality animated stickers usually contain transparent channels, which are often ignored by current video generation models. To generate fine-grained animated transparency channels, existing methods can be roughly divided into video matting algorithms and diffusion-based algorithms. The methods based on video matting have poor performance in dealing with semi-open areas in stickers, while diffusion-based methods are often used to model a single image, which will lead to local flicker when modeling animated stickers. In this paper, we firstly propose an ILDiff method to generate animated transparent channels through implicit layout distillation, which solves the problems of semi-open area collapse and no consideration of temporal information in existing methods. Secondly, we create the Transparent Animated Sticker Dataset (TASD), which contains 0.32M high-quality samples with transparent channel, to provide data support for related fields. Extensive experiments demonstrate that ILDiff can produce finer and smoother transparent channels compared to other methods such as Matting Anything and Layer Diffusion. Our code and dataset will be released at link https://xiaoyuan1996.github.io.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑