arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2407.08428cs.CVcs.AI
收藏

人体视频生成综合综述:挑战、方法与洞见

A Comprehensive Survey on Human Video Generation: Challenges, Methods, and Insights

Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州))

Wentao Lei, Jinting Wang, Fengji Ma, Guanjie Huang, Li Liu

AI总结:

该综述是首篇人体视频生成领域的大规模系统性综述,梳理了该任务的基础理论、生成模型演进、三大子任务方法、常用数据集与评估指标,总结了当前挑战与未来研究方向。

摘要:

人体视频生成是一项动态且快速发展的任务,旨在借助生成模型,在文本、音频、姿态等控制条件的输入下,合成二维人体视频序列。生成自然逼真的人体视频具备在影视、游戏、虚拟通信等领域的广泛应用潜力,因此该能力至关重要。近年来生成模型的技术进展,为该领域日益增长的研究热度奠定了坚实基础。尽管已取得显著进展,由于人物一致性要求、人体运动的复杂性,以及人体与环境交互关系的建模难点,人体视频生成任务仍面临诸多挑战。本综述对人体视频生成的研究现状进行了全面梳理,据我们所知,这是该领域首篇系统性的大规模文献综述。我们首先介绍人体视频生成的基础理论,以及推动该领域发展的生成模型演进历程。随后,我们围绕指导生成过程的控制条件,梳理了人体视频生成三大核心子任务的主要方法:文本驱动、音频驱动和姿态驱动的运动生成。此外,我们整理了该领域最常用的数据集集合,以及评估生成视频质量与真实感的关键评价指标。本综述最后讨论了该领域当前面临的挑战,并提出了未来研究的可行方向。本综述的目标是为研究界提供人体视频生成领域进展的清晰、整体视图,梳理已取得的里程碑成果与待解决的挑战。

英文摘要:

Human video generation is a dynamic and rapidly evolving task that aims to synthesize 2D human body video sequences with generative models given control conditions such as text, audio, and pose. With the potential for wide-ranging applications in film, gaming, and virtual communication, the ability to generate natural and realistic human video is critical. Recent advancements in generative models have laid a solid foundation for the growing interest in this area. Despite the significant progress, the task of human video generation remains challenging due to the consistency of characters, the complexity of human motion, and difficulties in their relationship with the environment. This survey provides a comprehensive review of the current state of human video generation, marking, to the best of our knowledge, the first extensive literature review in this domain. We start with an introduction to the fundamentals of human video generation and the evolution of generative models that have facilitated the field's growth. We then examine the main methods employed for three key sub-tasks within human video generation: text-driven, audio-driven, and pose-driven motion generation. These areas are explored concerning the conditions that guide the generation process. Furthermore, we offer a collection of the most commonly utilized datasets and the evaluation metrics that are crucial in assessing the quality and realism of generated videos. The survey concludes with a discussion of the current challenges in the field and suggests possible directions for future research. The goal of this survey is to offer the research community a clear and holistic view of the advancements in human video generation, highlighting the milestones achieved and the challenges that lie ahead.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑