Recently, Vector Quantized AutoRegressive (VQ-AR) models have shown remarkable results in text-to-image synthesis by equally predicting discrete image tokens from the top left to bottom right in the latent space. Although the simple generative process surprisingly works well, is this the best way to generate the image? For instance, human creation is more inclined to the outline-to-fine of an image, while VQ-AR models themselves do not consider any relative importance of image patches. In this paper, we present a progressive model for high-fidelity text-to-image generation. The proposed method takes effect by creating new image tokens from coarse to fine based on the existing context in a parallel manner, and this procedure is recursively applied with the proposed error revision mechanism until an image sequence is completed. The resulting coarse-to-fine hierarchy makes the image generation process intuitive and interpretable. Extensive experiments in MS COCO benchmark demonstrate that the progressive model produces significantly better results compared with the previous VQ-AR method in FID score across a wide variety of categories and aspects. Moreover, the design of parallel generation in each step allows more than $\times 13$ inference acceleration with slight performance loss.
Huazhong Univ. Sci. and Tech.（华中科技大学）；Microsoft（微软）；ZTE Corporation（中兴通讯股份有限公司）
This paper proposes a new generative adversarial network for pose transfer, i.e., transferring the pose of a given person to a target pose. The generator of the network comprises a sequence of Pose-Attentional Transfer Blocks that each transfers certain regions it attends to, generating the person image progressively. Compared with those in previous works, our generated person images possess better appearance consistency and shape consistency with the input images, thus significantly more realistic-looking. The efficacy and efficiency of the proposed network are validated both qualitatively an…
Twin Co-Adaptive Dialogue for Progressive Image Generation
University of Electronic Science and Technology of China（电子科技大学）；University of Minnesota Twin Cities（明尼苏达大学双城分校）；Peking University（北京大学）；Emory University（埃默里大学）；Xiamen University Malaysia（马来西亚厦门大学）；Hong Kong Polytechnic University（香港理工大学）；Tsinghua University（清华大学）；University of Warwick（沃里克大学）；Washington University, Saint Louis（圣路易斯华盛顿大学）；University of Toronto（多伦多大学）；Google（谷歌）
Progressive Human Motion Generation Based on Text and Few Motion Frames
School of Artificial Intelligence, Sun Yat-sen University（中山大学人工智能学院）；School of Computer Science and Engineering, Sun Yat-sen University（中山大学计算机科学与工程学院）；Peng Cheng Laboratory（鹏城实验室）
University of Illinois（伊利诺伊大学）；Huazhong University of Science and Technology（华中科技大学）；Microsoft（微软公司）
This paper proposes a new generative adversarial network for pose transfer, i.e., transferring the pose of a given person to a target pose. We design a progressive generator which comprises a sequence of transfer blocks. Each block performs an intermediate transfer step by modeling the relationship between the condition and the target poses with attention mechanism. Two types of blocks are introduced, namely Pose-Attentional Transfer Block (PATB) and Aligned Pose-Attentional Transfer Bloc ~(APATB). Compared with previous works, our model generates more photorealistic person images that retain …
Enhancing Image Generation Fidelity via Progressive Prompts
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；Tsinghua University（清华大学）；Peking university（北京大学）；EaseUS（易我科技）；The Hong Kong University of Science and Technology（香港科技大学）
This work presents a generative modeling approach based on successive subspace learning (SSL). Unlike most generative models in the literature, our method does not utilize neural networks to analyze the underlying source distribution and synthesize images. The resulting method, called the progressive attribute-guided extendable robust image generative (PAGER) model, has advantages in mathematical transparency, progressive content generation, lower training time, robust performance with fewer training samples, and extendibility to conditional image generation. PAGER consists of three modules: c…
CMD: Controllable Multiview Diffusion for 3D Editing and Progressive Generation
The Hong Kong University of Science and Technology（香港科学与技术大学）；Johns Hopkins University（约翰霍普金斯大学）；University of British Columbia（不列颠哥伦比亚大学）；Texas A\&M University（德克萨斯A&M大学）
TurboTalk: Progressive Distillation for One-Step Audio-Driven Talking Avatar Generation
MAIS, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所MAIS部）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Meituan（美团）；The Hong Kong Polytechnic University（香港理工大学）；CAIR, HKISI, CAS（中国科学院CAS HKISI CAIR）
Can We Build Scene Graphs, Not Classify Them? FlowSG: Progressive Image-Conditioned Scene Graph Generation with Flow Matching
The Laboratory of Intelligent Collaborative Computing of UESTC（UESTC智能协同计算实验室）；Tianfu Jiangxi Laboratory（天斧江西实验室）；Monash University（墨尔本大学）；Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ)（广东人工智能与数字经济实验室（深圳））
CERN（欧洲核子研究组织）；United Nations Satellite Centre (UNOSAT)（联合国卫星中心）
Image generation and image completion are rapidly evolving fields, thanks to machine learning algorithms that are able to realistically replace missing pixels. However, generating large high resolution images, with a large level of details, presents important computational challenges. In this work, we formulate the image generation task as completion of an image where one out of three corners is missing. We then extend this approach to iteratively build larger images with the same level of detail. Our goal is to obtain a scalable methodology to generate high resolution samples typically found …
Efficient Training for Human Video Generation with Entropy-Guided Prioritized Progressive Learning
Stanford University（斯坦福大学）；North China Electric Power University（华北电力大学）；University of Adelaide（阿德莱德大学）；University of Technology Sydney（悉尼技术大学）；University of Science and Technology of China（中国科学技术大学）
Pixel synthesis is a promising research paradigm for image generation, which can well exploit pixel-wise prior knowledge for generation. However, existing methods still suffer from excessive memory footprint and computation overhead. In this paper, we propose a progressive pixel synthesis network towards efficient image generation, coined as PixelFolder. Specifically, PixelFolder formulates image generation as a progressive pixel regression problem and synthesizes images via a multi-stage structure, which can greatly reduce the overhead caused by large tensor transformations. In addition, we i…
Generative adversarial networks (GANs) are widely used in image generation tasks, yet the generated images are usually lack of texture details. In this paper, we propose a general framework, called Progressively Unfreezing Perceptual GAN (PUPGAN), which can generate images with fine texture details. Particularly, we propose an adaptive perceptual discriminator with a pre-trained perceptual feature extractor, which can efficiently measure the discrepancy between multi-level features of the generated and real images. In addition, we propose a progressively unfreezing scheme for the adaptive perc…
We propose Progressive Structure-conditional Generative Adversarial Networks (PSGAN), a new framework that can generate full-body and high-resolution character images based on structural information. Recent progress in generative adversarial networks with progressive training has made it possible to generate high-resolution images. However, existing approaches have limitations in achieving both high image quality and structural consistency at the same time. Our method tackles the limitations by progressively increasing the resolution of both generated images and structural conditions during tr…
Department of Mathematics（数学系）；Computer Vision Laboratory（计算机视觉实验室）；Department of Information Technology and Electrical Engineering（信息技术与电气工程系）
The extension of image generation to video generation turns out to be a very difficult task, since the temporal dimension of videos introduces an extra challenge during the generation process. Besides, due to the limitation of memory and training stability, the generation becomes increasingly challenging with the increase of the resolution/duration of videos. In this work, we exploit the idea of progressive growing of Generative Adversarial Networks (GANs) for higher resolution video generation. In particular, we begin to produce video samples of low-resolution and short-duration, and then pro…
TunerDiT: Training-free Progressive Steering of Diffusion Transformer for Multi-Event Video Generation
Ludwig Maximilian University of Munich（慕尼黑路德维希-马克西米利安大学）；Technical University of Munich（慕尼黑技术大学）；MCML；University of Hamburg（汉堡大学）；Huawei European Research Institute（华为欧洲研究院）
Recent improvements in generative adversarial network (GAN) training techniques prove that progressively training a GAN drastically stabilizes the training and improves the quality of outputs produced. Adding layers after the previous ones have converged has proven to help in better overall convergence and stability of the model as well as reducing the training time by a sufficient amount. Thus we use this training technique to train the model progressively in the time and pitch domain i.e. starting from a very small time value and pitch range we gradually expand the matrix sizes until the end…
Progressive Rendering Distillation: Adapting Stable Diffusion for Instant Text-to-Mesh Generation without 3D Data
The Hong Kong Polytechnic University（香港理工大学）；Center for Artificial Intelligence and Robotics, HKISI CAS（香港科学院人工智能与机器人中心）；State Key Laboratory of Multimodal Artificial Intelligence Systems, CASIA（中国科学院自动化研究所多模态人工智能系统国家重点实验室）；School of Artificial Intelligence, University of Chinese Academy of Sciences, UCAS（中国科学院大学人工智能学院）
Are We Ready for RL in Text-to-3D Generation? A Progressive Investigation
Northwestern Polytechnical University（西北工业大学）；Peking University（北京大学）；The Chinese University of Hong Kong（香港中文大学）；Shanghai AI Lab（上海人工智能实验室）；The Hong Kong University of Science and Technology（香港科技大学）
Nanjing University of Science and Technology（南京理工大学）
In this paper, we propose an effective point cloud generation method, which can generate multi-resolution point clouds of the same shape from a latent vector. Specifically, we develop a novel progressive deconvolution network with the learning-based bilateral interpolation. The learning-based bilateral interpolation is performed in the spatial and feature spaces of point clouds so that local geometric structure information of point clouds can be exploited. Starting from the low-resolution point clouds, with the bilateral interpolation and max-pooling operations, the deconvolution network can p…
With the development of 3D scanning technologies, 3D vision tasks have become a popular research area. Owing to the large amount of data acquired by sensors, unsupervised learning is essential for understanding and utilizing point clouds without an expensive annotation process. In this paper, we propose a novel framework and an effective auto-encoder architecture named "PSG-Net" for reconstruction-based learning of point clouds. Unlike existing studies that used fixed or random 2D points, our framework generates input-dependent point-wise features for the latent point set. PSG-Net uses the enc…
National Key Lab for Novel Software Technology, Nanjing University（南京大学计算机科学与技术系国家重点实验室（注：此处原机构名中“National Key Lab for Novel Software Technology”是南京大学的国家重点实验室，中文通常译为“南京大学计算机科学与技术系国家重点实验室”，但根据要求只翻译机构名称本身，此处按原英文机构名直译，若需更准确的中文名，需结合具体研究方向。不过根据题目要求，保留英文机构名同时给出中文翻译，此处“National Key Lab for Novel Software Technology, Nanjing University”的常见中文译名为“南京大学计算机科学与技术系国家重点实验室”，但更准确的是“南京大学计算机软件新技术国家重点实验室”，因“Novel Software Technology”通常译为“计算机软件新技术”。为更准确，调整为“南京大学计算机软件新技术国家重点实验室”））
The challenges of shape robust text detection lie in two aspects: 1) most existing quadrangular bounding box based detectors are difficult to locate texts with arbitrary shapes, which are hard to be enclosed perfectly in a rectangle; 2) most pixel-wise segmentation-based detectors may not separate the text instances that are very close to each other. To address these problems, we propose a novel Progressive Scale Expansion Network (PSENet), designed as a segmentation-based detector with multiple predictions for each text instance. These predictions correspond to different `kernels' produced by…
