FairyGen：从单张儿童手绘角色生成故事驱动的卡通视频
FairyGen: Storied Cartoon Video from a Single Child-Drawn Character
AI总结：
FairyGen系统从单张儿童画作生成故事卡通视频，通过解耦角色与背景生成、引入电影镜头设计及两阶段运动定制适配器，实现了系统实现了风格忠实、叙事连贯且运动自然的动画生成。
摘要：
我们提出了FairyGen，一个从单张儿童画作自动生成故事驱动卡通视频的系统，同时忠实保留其独特的艺术风格。与以往主要关注角色一致性和基本运动的故事讲述方法不同，FairyGen明确地将角色建模与风格化背景生成解耦，并引入电影镜头设计以支持富有表现力且连贯的故事讲述。给定单张角色草图，我们首先采用MLLM生成结构化故事板，包含指定环境设置、角色动作和摄像机视角的镜头级描述。为确保视觉一致性，我们引入风格传播适配器，捕获角色视觉风格并应用于背景，在合成风格一致场景时忠实保留角色完整视觉特征。镜头设计模块通过基于故事板的帧裁剪和多视图合成，进一步增强视觉多样性和电影质感。为使故事动起来，我们重建角色3D代理模型以推导物理合理的运动序列，随后用于微调基于MMDiT的图生视频扩散模型。我们进一步提出两阶段运动定制适配器：第一阶段从时间无序帧学习外观特征，将身份与运动解耦；第二阶段使用时间步偏移策略并冻结身份权重来建模时间动态。训练完成后，FairyGen直接渲染与故事板对齐的多样且连贯的视频场景。大量实验表明，本系统生成的动画在风格上忠实、叙事结构化且运动自然，突显其在个性化且引人入胜的故事动画方面的潜力。代码将发布于https://github.com/GVCLab/FairyGen
英文摘要：
We propose FairyGen, an automatic system for generating story-driven cartoon videos from a single child's drawing, while faithfully preserving its unique artistic style. Unlike previous storytelling methods that primarily focus on character consistency and basic motion, FairyGen explicitly disentangles character modeling from stylized background generation and incorporates cinematic shot design to support expressive and coherent storytelling. Given a single character sketch, we first employ an MLLM to generate a structured storyboard with shot-level descriptions that specify environment settings, character actions, and camera perspectives. To ensure visual consistency, we introduce a style propagation adapter that captures the character's visual style and applies it to the background, faithfully retaining the character's full visual identity while synthesizing style-consistent scenes. A shot design module further enhances visual diversity and cinematic quality through frame cropping and multi-view synthesis based on the storyboard. To animate the story, we reconstruct a 3D proxy of the character to derive physically plausible motion sequences, which are then used to fine-tune an MMDiT-based image-to-video diffusion model. We further propose a two-stage motion customization adapter: the first stage learns appearance features from temporally unordered frames, disentangling identity from motion; the second stage models temporal dynamics using a timestep-shift strategy with frozen identity weights. Once trained, FairyGen directly renders diverse and coherent video scenes aligned with the storyboard. Extensive experiments demonstrate that our system produces animations that are stylistically faithful, narratively structured natural motion, highlighting its potential for personalized and engaging story animation. The code will be available at https://github.com/GVCLab/FairyGen
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
