BiGR: Harnessing Binary Latent Codes for Image Generation and Improved Visual Representation Capabilities
The University of Hong Kong（香港大学）；Hong Kong University of Science and Technology（香港科技大学）；Intellifusion（云天励飞）；The Chinese University of Hong Kong（香港中文大学）
Shaozhe Hao, Xuantong Liu, Xianbiao Qi, Shihao Zhao, Bojia Zi, Rong Xiao, Kai Han, Kwan-Yee K. Wong
We introduce BiGR, a novel conditional image generation model using compact binary latent codes for generative training, focusing on enhancing both generation and representation capabilities. BiGR is the first conditional generative model that unifies generation and discrimination within the same framework. BiGR features a binary tokenizer, a masked modeling mechanism, and a binary transcoder for binary code prediction. Additionally, we introduce a novel entropy-ordered sampling method to enable efficient image generation. Extensive experiments validate BiGR's superior performance in generation quality, as measured by FID-50k, and representation capabilities, as evidenced by linear-probe accuracy. Moreover, BiGR showcases zero-shot generalization across various vision tasks, enabling applications such as image inpainting, outpainting, editing, interpolation, and enrichment, without the need for structural modifications. Our findings suggest that BiGR unifies generative and discriminative tasks effectively, paving the way for further advancements in the field. We further enable BiGR to perform text-to-image generation, showcasing its potential for broader applications.
Adversarial examples are fabricated examples, indistinguishable from the original image that mislead neural networks and drastically lower their performance. Recently proposed AdvGAN, a GAN based approach, takes input image as a prior for generating adversaries to target a model. In this work, we show how latent features can serve as better priors than input images for adversary generation by proposing AdvGAN++, a version of AdvGAN that achieves higher attack rates than AdvGAN and at the same time generates perceptually realistic images on MNIST and CIFAR-10 datasets.
Focusing on What Matters: Saliency-Harnessing Accurate Routing for Diffusion MoE
Key Laboratory of Image Processing and Intelligent Control, School of Artificial Intelligence and Automation, Huazhong University of Science and Technology（华中科技大学人工智能与自动化学院图像处理与智能控制重点实验室）；Tongyi Lab, Alibaba Group（阿里巴巴集团通义实验室）
Derm-T2IM: Harnessing Synthetic Skin Lesion Data via Stable Diffusion Models for Enhanced Skin Disease Classification using ViT and CNN
College of Science and Engineering, University of Galway（戈尔韦大学科学与工程学院）；School of Computer Science, University of Galway（戈尔韦大学计算机科学学院）；Trinity College Dublin, the University of Dublin（都柏林大学圣三一学院）；St James’s Hospital（圣詹姆斯医院）
本文提出 Derm-T2IM，用调优的 Stable Diffusion 生成多样化皮肤镜病变图像，并验证其可增强 CNN 与 ViT 在两个真实数据集上的分类鲁棒性和泛化性。
Neural networks are vulnerable to adversarial attacks -- small visually imperceptible crafted noise which when added to the input drastically changes the output. The most effective method of defending against these adversarial attacks is to use the methodology of adversarial training. We analyze the adversarially trained robust models to study their vulnerability against adversarial attacks at the level of the latent layers. Our analysis reveals that contrary to the input layer which is robust to adversarial attack, the latent layer of these robust models are highly susceptible to adversarial …
Macau University of Science and Technology（澳门科技大学）；Communication University of Zhejiang（浙江传媒学院）；Zhejiang Lab（之江实验室）；Communication University of China（中国传媒大学）；Zhejiang University（浙江大学）；Zhejiang Institute of Economics and Trade（浙江经贸职业技术学院）
BOOD: Boundary-based Out-Of-Distribution Data Generation
The University of Hong Kong, Hong Kong, China（香港大学）；School of AI, Shanghai Jiao Tong University, Shanghai, China（上海交通大学人工智能学院）；Department of Computer Science, Harbin Institute of Technology (Shenzhen), Shenzhen, China（哈尔滨工业大学（深圳）计算机科学系）
Representation Entanglement for Generation: Training Diffusion Transformers Is Much Easier Than You Think
NKIARI, Shenzhen Futian（NKIARI，深圳福田）；VCIP, CS, Nankai University（VCIP，计算机科学，南开大学）；JIIOV Technology（JIIOV技术）；Harvard University（哈佛大学）；University of Chinese Academy of Sciences（中国科学院大学）
The common implementation of face recognition systems as a cascade of a detection stage and a recognition or verification stage can cause problems beyond failures of the detector. When the detector succeeds, it can detect faces that cannot be recognized, no matter how capable the recognition system. Recognizability, a latent variable, should therefore be factored into the design and implementation of face recognition systems. We propose a measure of recognizability of a face image that leverages a key empirical observation: an embedding of face images, implemented by a deep neural network trai…
Harnessing Intra-group Variations Via a Population-Level Context for Pathology Detection
Tsinghua University（清华大学）；Tsinghua-Berkeley Shenzhen Institute（清华-伯克利深圳学院）；School of Communication and Information Engineering, Shanghai University（上海大学通信与信息工程学院）；School of Mechanical, Electrical & Information Engineering, Shandong University（山东大学机电与信息工程学院）；School of Automation, Hangzhou Dianzi University（杭州电子科技大学自动化学院）
Binary classification (BC) is a practical task that is ubiquitous in real-world problems, such as distinguishing healthy and unhealthy objects in biomedical diagnostics and defective and non-defective products in manufacturing inspections. Nonetheless, fully annotated data are commonly required to effectively solve this problem, and their collection by domain experts is a tedious and expensive procedure. In contrast to BC, several significant semi-supervised learning techniques that heavily rely on stochastic data augmentation techniques have been devised for solving multi-class classification…
Center for Computational Mathematics, Flatiron Institute, New York, NY, USA；Department of Statistics, University of Washington, Seattle, Washington, USA；Frankfurt Institute For Advanced Study, Frankfurt, Hesse, Germany；Center for Computational Biology, Flatiron Institute, New York, NY, USA；Institute of Computer Science, Goethe University Frankfurt, Frankfurt, Hesse, Germany
