Image rescaling aims to learn the optimal low-resolution (LR) image that can be accurately reconstructed to its original high-resolution (HR) counterpart, providing an efficient image processing and storage method for ultra-high definition media. However, extreme downscaling factors pose significant challenges to the upscaling process due to its highly ill-posed nature, causing existing image rescaling methods to struggle in generating semantically correct structures and perceptual friendly textures. In this work, we propose a novel framework called Timestep-Aware Diffusion Model (TADM) for extreme image rescaling, which performs rescaling operations in the latent space of a pre-trained autoencoder and effectively leverages powerful natural image priors learned by a pre-trained text-to-image diffusion model. Specifically, TADM adopts a pseudo-invertible module to establish the bidirectional mapping between the latent features of the HR image and the target-sized LR image. Then, the rescaled latent features are enhanced by a pre-trained diffusion model to generate more faithful details. Considering the spatially non-uniform degradation caused by the rescaling operation, we propose a novel time-step alignment strategy, which can adaptively allocate the generative capacity of the diffusion model based on the quality of the reconstructed latent features. Extensive experiments demonstrate the superiority of TADM over previous methods in both quantitative and qualitative evaluations.
Timestep Embedding Tells: It's Time to Cache for Video Diffusion Model
University of Chinese Academy of Sciences（中国科学院大学）；Alibaba Group（阿里巴巴集团）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Fudan University（复旦大学）；Nanyang Technological University（南洋理工大学）
Faithful Extreme Image Rescaling with Learnable Reversible Transformation and Semantic Priors
State Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University（人机混合增强智能国家重点实验室，人工智能与机器人研究院，西安交通大学）；School of Information and Communications Engineering, Xi’an Jiaotong University（信息与通信工程学院，西安交通大学）；Department of Computer Science and Software Engineering, The University of Western Australia（计算机科学与软件工程系，西澳大学）
Boundary-aware Decoupled Flow Networks for Realistic Extreme Rescaling
Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；College of Computer Science and Software Engineering, Shenzhen University（深圳大学计算机与软件学院）；WeChat Pay Lab33, Tencent（腾讯微信支付实验室）；Research Center of Artificial Intelligence, Peng Cheng Laboratory（鹏城实验室人工智能研究中心）
Joint Degradation-Aware Arbitrary-Scale Super-Resolution for Variable-Rate Extreme Image Compression
School of Information Science and Electronic Engineering, Shanghai Jiao Tong University（上海交通大学信息科学与电子工程学院）；Department of Electronic Engineer and Information Science, University of Science and Technology of China（中国科学技术大学电子工程与信息科学系）；MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University（上海交通大学人工智能教育部重点实验室）
Hierarchical Schedule Optimization for Fast and Robust Diffusion Model Sampling
School of Computer Science and Engineering, Macau University of Science and Technology（澳门科学技术大学计算机科学与工程学院）；Beijing Institute of Technology（北京理工大学）；Zhejiang University（浙江大学）
GenMask: Adapting DiT for Segmentation via Direct Mask Generation
Cooperative Medianet Innovation Center, Shanghai Jiao Tong University（上海交通大学协作中位网创新中心）；School of Artificial Intelligence, Shanghai Jiao Tong University（上海交通大学人工智能学院）；Institute of Artificial Intelligence for Medicine, Shanghai Jiao Tong University School of Medicine（上海交通大学医学院人工智能医学研究院）；Alibaba Group（阿里巴巴集团）
Diffusion-based generative models have achieved remarkable success in image generation. Their guidance formulation allows an external model to plug-and-play control the generation process for various tasks without finetuning the diffusion model. However, the direct use of publicly available off-the-shelf models for guidance fails due to their poor performance on noisy inputs. For that, the existing practice is to fine-tune the guidance models with labeled data corrupted with noises. In this paper, we argue that this practice has limitations in two aspects: (1) performing on inputs with extreme…
Gungnir: Exploiting Stylistic Features in Images for Backdoor Attacks on Diffusion Models
School of Economics and Management（经济管理学院）；East China Normal University（华东师范大学）；School of Information Science and Technology（信息科学与技术学院）；Shanghai Tech University（上海理工大学）；Shanghai Development Center of Computer Software Technology（上海计算机软件技术开发中心）；School of Computer and Information Engineering（计算机与信息工程学院）；Shanghai Polytechnic University（上海理工大学）
Spatiotemporally adaptive compression for scientific dataset with feature preservation -- a case study on simulation data with extreme climate events analysis
Oak Ridge National Laboratory（橡树岭国家实验室）；Lawrence Livermore National Laboratory（劳伦斯利弗莫尔国家实验室）；University of Kentucky（肯塔基大学）；University of Alabama（阿拉巴马大学）；University of Florida（佛罗里达大学）；Georgia State University（佐治亚州立大学）；Argonne National Laboratory（阿贡国家实验室）
Discrete Diffusion Bridges for Spatiotemporally Aligned Image Translation and Generation
Shenyang Institute of Automation, Chinese Academy of Sciences（中国科学院沈阳自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；The University of Hong Kong（香港大学）；School of Computing and Data Science, The University of Hong Kong（香港大学计算与数据科学学院）
