Diffusion-based image synthesis has attracted extensive attention recently. In particular, ControlNet that uses image-based prompts exhibits powerful capability in image tasks such as canny edge detection and generates images well aligned with these prompts. However, vanilla ControlNet generally requires extensive training of around 5000 steps to achieve a desirable control for a single task. Recent context-learning approaches have improved its adaptability, but mainly for edge-based tasks, and rely on paired examples. Thus, two important open issues are yet to be addressed to reach the full potential of ControlNet: (i) zero-shot control for certain tasks and (ii) faster adaptation for non-edge-based tasks. In this paper, we introduce a novel Meta ControlNet method, which adopts the task-agnostic meta learning technique and features a new layer freezing design. Meta ControlNet significantly reduces learning steps to attain control ability from 5000 to 1000. Further, Meta ControlNet exhibits direct zero-shot adaptability in edge-based tasks without any finetuning, and achieves control within only 100 finetuning steps in more complex non-edge tasks such as Human Pose, outperforming all existing methods. The codes is available in https://github.com/JunjieYang97/Meta-ControlNet.
PIXART-δ: Fast and Controllable Image Generation with Latent Consistency Models
Huawei Noah’s Ark Lab（华为诺亚方舟实验室）；Dalian University of Technology（大连理工大学）；The University of Hong Kong（香港大学）；IIIS, Tsinghua University（清华大学智能产业研究院）；Hugging Face
University of British Columbia（不列颠哥伦比亚大学）；Amazon（亚马逊）；Northeastern University（东北大学）；Inverted AI；MILA（蒙特利尔学习算法研究所）；Beam AI
We develop a transductive meta-learning method that uses unlabelled instances to improve few-shot image classification performance. Our approach combines a regularized Mahalanobis-distance-based soft k-means clustering procedure with a modified state of the art neural adaptive feature extractor to achieve improved test-time classification accuracy using unlabelled data. We evaluate our method on transductive few-shot learning tasks, in which the goal is to jointly predict labels for query (test) examples given a set of support (training) examples. We achieve state of the art performance on the…
Adaptive Clipping for Privacy-Preserving Few-Shot Learning: Enhancing Generalization with Limited Data
Deakin University（迪肯大学）；Data61, CSIRO（联邦科学与工业研究组织Data61研究院）；The University of Alabama in Huntsville（阿拉巴马大学亨茨维尔分校）；University of New South Wales (UNSW)（新南威尔士大学）
