arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2206.12396cs.CV
收藏

文本驱动的视频对象风格化

Text-Driven Stylization of Video Objects

Aarhus University(奥胡斯大学);University of Copenhagen(哥本哈根大学)

Sebastian Loeschcke, Serge Belongie, Sagie Benaim

摘要:

我们处理的任务,风格化的视频对象在一个直观的和语义的方式后,用户指定的文本提示。这是一项具有挑战性的任务,因为所得到的视频必须满足多个属性:(1)它必须在时间上一致,并避免抖动或类似的伪影,(2)所得到的风格化必须保留对象的全局语义及其细粒度细节,以及(3)它必须遵守用户指定的文本提示。为此,我们的方法根据两个目标文本对视频中的对象进行风格化。第一个目标文本提示符描述全局语义,第二个目标文本提示符描述局部语义。为了修改对象的样式,我们利用CLIP的表示能力来获得(1)局部目标文本和一组局部风格化视图之间的相似性得分,以及(2)全局目标文本和一组风格化全局视图之间的相似性得分。我们使用预训练的图谱分解网络以时间一致的方式传播编辑。我们证明,我们的方法可以生成一致的风格变化随着时间的推移,各种对象和视频,坚持目标文本的规范。我们还展示了如何改变目标文本的特异性和增强文本与一组前缀的结果在风格化与不同层次的细节。完整的结果在我们的项目网页:https://sloeschcke.github.io/Text-Driven-Stylization-of-Video-Objects/

英文摘要:

We tackle the task of stylizing video objects in an intuitive and semantic manner following a user-specified text prompt. This is a challenging task as the resulting video must satisfy multiple properties: (1) it has to be temporally consistent and avoid jittering or similar artifacts, (2) the resulting stylization must preserve both the global semantics of the object and its fine-grained details, and (3) it must adhere to the user-specified text prompt. To this end, our method stylizes an object in a video according to two target texts. The first target text prompt describes the global semantics and the second target text prompt describes the local semantics. To modify the style of an object, we harness the representational power of CLIP to get a similarity score between (1) the local target text and a set of local stylized views, and (2) a global target text and a set of stylized global views. We use a pretrained atlas decomposition network to propagate the edits in a temporally consistent manner. We demonstrate that our method can generate consistent style changes over time for a variety of objects and videos, that adhere to the specification of the target texts. We also show how varying the specificity of the target texts and augmenting the texts with a set of prefixes results in stylizations with different levels of detail. Full results are given on our project webpage: https://sloeschcke.github.io/Text-Driven-Stylization-of-Video-Objects/

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑