EVATok: 适应性长度视频分块化以实现高效的视觉自回归生成
EVATok: Adaptive Length Video Tokenization for Efficient Visual Autoregressive Generation
AI总结：
EVATok通过自适应分块器提升视频重建和自回归生成的效率与质量，实现令牌使用量减少24.4%。
摘要：
自回归（AR）视频生成模型依赖于将像素压缩成离散令牌序列的视频分块器。这些令牌序列的长度对于在重建质量与下游生成计算成本之间取得平衡至关重要。传统视频分块器在不同视频的时序块中采用统一的令牌分配，常常在简单的、静态的或重复的片段上浪费令牌，而忽视动态或复杂的片段。为了解决这种低效性，我们引入了EVATok，一个生成高效视频自适应分块器的框架。我们的框架为每个视频估计最佳的令牌分配，以实现最佳的质量-成本折中，开发轻量级的路由器以快速预测这些最佳分配，并训练自适应分块器，根据路由器预测的分配来编码视频。我们证明，EVATok在视频重建和下游AR生成的效率和整体质量上均取得了显著改进。通过我们先进的训练配方，整合了视频语义编码器，EVATok在UCF-101上实现了优越的重建和最先进的类到视频生成，与先前最先进的LARP和我们的固定长度基线相比，平均令牌使用量至少减少了24.4%。
英文摘要：
Autoregressive (AR) video generative models rely on video tokenizers that compress pixels into discrete token sequences. The length of these token sequences is crucial for balancing reconstruction quality against downstream generation computational cost. Traditional video tokenizers apply a uniform token assignment across temporal blocks of different videos, often wasting tokens on simple, static, or repetitive segments while underserving dynamic or complex ones. To address this inefficiency, we introduce $\textbf{EVATok}$, a framework to produce $\textbf{E}$fficient $\textbf{V}$ideo $\textbf{A}$daptive $\textbf{Tok}$enizers. Our framework estimates optimal token assignments for each video to achieve the best quality-cost trade-off, develops lightweight routers for fast prediction of these optimal assignments, and trains adaptive tokenizers that encode videos based on the assignments predicted by routers. We demonstrate that EVATok delivers substantial improvements in efficiency and overall quality for video reconstruction and downstream AR generation. Enhanced by our advanced training recipe that integrates video semantic encoders, EVATok achieves superior reconstruction and state-of-the-art class-to-video generation on UCF-101, with at least 24.4% savings in average token usage compared to the prior state-of-the-art LARP and our fixed-length baseline.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
