FaceCam: 通过尺度感知条件化实现人像视频相机控制
FaceCam: Portrait Video Camera Control via Scale-Aware Conditioning
AI总结：
FaceCam通过尺度感知条件化方法，提升单目人像视频的相机控制能力与视觉质量。
摘要：
我们介绍了FaceCam，一个能够根据可定制的相机轨迹生成视频的系统，用于单目人像视频输入。最近基于大视频生成模型的相机控制方法在理论上取得了显著进展，但往往在人像视频上表现出几何失真和视觉伪影，这是由于尺度模糊的相机表示或3D重建误差所致。为克服这些限制，我们提出了一种面向面部的尺度感知表示，用于相机变换，该表示在不依赖3D先验的情况下提供确定性条件化。我们训练了一个视频生成模型，使用多视角摄影棚拍摄和野外单目视频进行训练，并引入了两种相机控制数据生成策略：合成相机运动和多镜头拼接，以利用静止训练相机的同时，在推理时泛化到动态、连续的相机轨迹。在Ava-256数据集和多样化的野外视频上的实验表明，FaceCam在相机可控性、视觉质量、身份和运动保持方面均取得了优越的性能。
英文摘要：
We introduce FaceCam, a system that generates video under customizable camera trajectories for monocular human portrait video input. Recent camera control approaches based on large video-generation models have shown promising progress but often exhibit geometric distortions and visual artifacts on portrait videos due to scale-ambiguous camera representations or 3D reconstruction errors. To overcome these limitations, we propose a face-tailored scale-aware representation for camera transformations that provides deterministic conditioning without relying on 3D priors. We train a video generation model on both multi-view studio captures and in-the-wild monocular videos, and introduce two camera-control data generation strategies: synthetic camera motion and multi-shot stitching, to exploit stationary training cameras while generalizing to dynamic, continuous camera trajectories at inference time. Experiments on Ava-256 dataset and diverse in-the-wild videos demonstrate that FaceCam achieves superior performance in camera controllability, visual quality, identity and motion preservation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
