BioVid: 具有生物行为语义理解的自回归视频生成
BioVid: Autoregressive Video Generation with Biological Behavior Semantic Comprehension
AI总结：
提出BioVid，一种数据驱动的自回归视频生成框架，通过FSQ-R3GAN分词器和因果Transformer学习生物行为的自然时长分布，无需预设长度约束。
摘要：
现有的视频生成框架将序列时长视为外部指定参数——固定的帧数或文本提示——生成的片段在时间边界上与真实行为数据的统计结构脱节。这一假设与生物行为根本不一致，因为动作时长在个体和实例之间自然变化，并编码在数据本身中。我们提出BioVid，一种数据驱动的自回归视频生成框架，直接从训练数据中学习生物行为的时序结构，包括其自然长度分布。在第一阶段，有限标量量化GAN（FSQ-R3GAN）分词器将每个视频帧编码为紧凑的离散表示，结合R3GAN的稳定相对训练目标和FSQ的保证码本利用率，实现高保真空间重建而无需码本崩溃。在第二阶段，因果Transformer自回归地对生成的令牌序列建模，并在行为事件达到语义闭合时学习发出序列结束（EOS）令牌，终止分布自然地从训练数据中涌现，而非任何人为指定的约束。在人类饮酒行为数据集（NTU RGB+D, A001, n=94）上的实验表明，BioVid生成的长度分布与保留测试数据的分布紧密匹配，与真实分布的Wasserstein-1距离为1.24——相比之下，固定长度基线为6.05，VideoGPT为15.48——同时保持有竞争力的空间保真度。
英文摘要：
Video generation for biological behavior requires more than visually plausible motion: the duration of an action is itself a semantic property. Existing models usually rely on fixed temporal windows, external continuation, or prompt-driven stories, so length is specified externally rather than learned from behavior. To address this gap, we propose BioVid, a data-driven autoregressive framework for adaptive-length biological behavior generation. BioVid uses a 2D-encode/3D-decode tokenizer: a two-dimensional FSQ-R3GAN encoder converts each frame into discrete visual tokens, preserving single-frame information suited for next-token prediction and EOS-based termination, while a temporally inflated and video-finetuned three-dimensional decoder reconstructs generated tokens with temporal context to reduce flickering. A causal Transformer then models the frame-wise token sequence and, conditioned only on the first frame, stops generation when it emits an End-of-Sequence token, allowing duration to emerge from the learned behavior distribution. We evaluate BioVid on the A001 drinking action from NTU RGB+D. On 94 held-out clips, BioVid achieves a Wasserstein-1 distance of 1.24 frames from the real duration distribution. In comparison, fixed-length baselines yield distances of approximately 6-7 frames even when configured to the available length closest to the dataset mean, and approximately 15 frames when using the conventional 16-frame generation length. These results demonstrate the ability of BioVid to learn and reproduce the intrinsic duration distribution of biological behavior.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
