MEPA: 基于专家混合的多尺度表示对齐用于视觉自回归建模
MEPA: Multi-Scale Representation Alignment for Visual Autoregressive Modeling with Mixture of Experts
AI总结：
针对视觉自回归模型在多尺度表示学习中的优化冲突和语义传播问题，提出尺度感知的专家混合架构和残差特征聚合方案，提升训练效率和生成质量。
摘要：
视觉自回归建模（VAR）开创了从粗到细的多尺度自回归生成范式，在图像生成中展现出强大能力。然而，VAR在多尺度表示学习中仍存在固有缺陷。具体而言，较低尺度主要捕获全局语义，而较高尺度关注细粒度细节。跨尺度采用共享架构会导致优化冲突。此外，由于因果自回归过程，早期尺度的不准确语义会传播并显著降低最终输出。为解决这些问题，我们引入了一种尺度感知的令牌路由专家混合（MoE）架构，允许尺度自适应的专家选择，从而促进跨尺度的解耦表示学习。此外，我们通过引入外部自监督特征来增强早期尺度的语义建模。与朴素对齐不同，我们分析并设计了一种针对VAR范式的残差特征聚合方案。大量实验表明，我们的方法显著提升了训练效率和生成质量。在ImageNet 256*256基准上，我们的模型相比密集基线实现了更优的FID，同时仅需一半的默认训练轮次和更小的参数预算，且训练成本仅略有增加。此外，随着训练轮次的增加，性能差距进一步扩大。
英文摘要：
Visual AutoRegressive modeling (VAR) has pioneered a coarse-to-fine multi-scale autoregressive generative paradigm, demonstrating strong capabilities in image generation. However, VAR still suffers from inherent deficiencies in multi-scale representation learning. Specifically, lower scales primarily capture global semantics, while higher scales focus on fine-grained details. Employing a shared architecture across scales induces optimization conflicts. Moreover, due to the causal autoregressive process, inaccurate semantics at early scales can propagate and significantly degrade the final output. To address these issues, we introduce a scale-aware token-routed Mixture of Experts (MoE) architecture, allowing scale-adaptive expert selection, thereby facilitating decoupled representation learning across scales. In addition, we enhance semantic modeling at early scales by incorporating external self-supervised features. Unlike naive alignment, we analyse and design a residual feature aggregation scheme tailored to the VAR paradigm. Extensive experiments show that our method significantly improves both training efficiency and generation quality. On the ImageNet 256*256 benchmark, our model achieves a superior FID compared to the dense baseline while requiring only half of the default training epochs and a smaller parameter budget, with a merely marginal increase in training cost. Moreover, the performance gap further widens with larger training epochs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
