HOFAR: High-Order Augmentation of Flow Autoregressive Transformers
The University of Hong Kong（香港大学）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；Tsinghua University（清华大学）；The Simons Institute for the Theory of Computing at UC Berkeley（加州大学伯克利分校西蒙斯理论计算研究所）；Anhui University（安徽大学）
Yingyu Liang, Zhizhou Sha, Zhenmei Shi, Zhao Song, Mingda Wan
流匹配（Flow Matching）与Transformer架构在图像生成任务中已展现出卓越性能，近期工作FlowAR [Ren et al., 2024]协同整合了这两种范式以提升合成保真度。然而，当前FlowAR实现仍受限于生成过程中的一阶轨迹建模。本文提出一种新颖框架，通过高阶监督系统性地增强流自回归变换器。我们提供了理论分析和实证评估，表明我们的高阶FlowAR（HOFAR）与基线模型相比在生成质量上展现出可测量的改进。所提方法通过引入一个基于高阶展开分析轨迹动力学的系统性框架，推进了对基于流的自回归建模的理解。
英文摘要：
Flow Matching and Transformer architectures have demonstrated remarkable performance in image generation tasks, with recent work FlowAR [Ren et al., 2024] synergistically integrating both paradigms to advance synthesis fidelity. However, current FlowAR implementations remain constrained by first-order trajectory modeling during the generation process. This paper introduces a novel framework that systematically enhances flow autoregressive transformers through high-order supervision. We provide theoretical analysis and empirical evaluation showing that our High-Order FlowAR (HOFAR) demonstrates measurable improvements in generation quality compared to baseline models. The proposed approach advances the understanding of flow-based autoregressive modeling by introducing a systematic framework for analyzing trajectory dynamics through high-order expansion.
Video prediction is an important yet challenging problem; burdened with the tasks of generating future frames and learning environment dynamics. Recently, autoregressive latent video models have proved to be a powerful video prediction tool, by separating the video prediction into two sub-problems: pre-training an image generator model, followed by learning an autoregressive prediction model in the latent space of the image generator. However, successfully generating high-fidelity and high-resolution videos has yet to be seen. In this work, we investigate how to train an autoregressive latent …
Korea University（高丽大学）；Drexel University（德雷塞尔大学）；DMLab
In this paper, a deep learning-based model for 3D human motion generation from the text is proposed via gesture action classification and an autoregressive model. The model focuses on generating special gestures that express human thinking, such as waving and nodding. To achieve the goal, the proposed method predicts expression from the sentences using a text classification model based on a pretrained language model and generates gestures using the gate recurrent unit-based autoregressive model. Especially, we proposed the loss for the embedding space for restoring raw motions and generating i…
We explore deep autoregressive Transformer models in language modeling for speech recognition. We focus on two aspects. First, we revisit Transformer model configurations specifically for language modeling. We show that well configured Transformer models outperform our baseline models based on the shallow stack of LSTM recurrent neural network layers. We carry out experiments on the open-source LibriSpeech 960hr task, for both 200K vocabulary word-level and 10K byte-pair encoding subword-level language modeling. We apply our word-level models to conventional hybrid speech recognition by lattic…
LINA: Linear Autoregressive Image Generative Models with Continuous Tokens
The University of Hong Kong（香港大学）；University of Chinese Academy of Sciences（中国科学院大学）；Tsinghua University（清华大学）；Beijing Academy of Artificial Intelligence（北京人工智能研究院）
University of Surrey（萨里大学）；iFlyTek-Surrey Joint Research Centre on Artificial Intelligence（科大讯飞-萨里人工智能联合研究中心）
Given an abstract, deformed, ordinary sketch from untrained amateurs like you and me, this paper turns it into a photorealistic image - just like those shown in Fig. 1(a), all non-cherry-picked. We differ significantly from prior art in that we do not dictate an edgemap-like sketch to start with, but aim to work with abstract free-hand human sketches. In doing so, we essentially democratise the sketch-to-photo pipeline, "picturing" a sketch regardless of how good you sketch. Our contribution at the outset is a decoupled encoder-decoder training paradigm, where the decoder is a StyleGAN trained…
LaGen: Towards Autoregressive LiDAR Scene Generation
University of Science and Technology of China（中国科学技术大学）；Shanghai Innovation Institute（上海创新研究院）；Shanghai Jiao Tong University（上海交通大学）；Fudan University（复旦大学）
