We develop a transductive meta-learning method that uses unlabelled instances to improve few-shot image classification performance. Our approach combines a regularized Mahalanobis-distance-based soft k-means clustering procedure with a modified state of the art neural adaptive feature extractor to achieve improved test-time classification accuracy using unlabelled data. We evaluate our method on transductive few-shot learning tasks, in which the goal is to jointly predict labels for query (test) examples given a set of support (training) examples. We achieve state of the art performance on the Meta-Dataset, mini-ImageNet and tiered-ImageNet benchmarks. All trained models and code have been made publicly available at github.com/plai-group/simple-cnaps.
iFlyTek-Surrey Joint Research Centre on Artificial Intelligence（科大讯飞-萨里人工智能联合研究中心）
Few shot learning aims to solve the data scarcity problem. If there is a domain shift between the test set and the training set, their performance will decrease a lot. This setting is called Cross-domain few-shot learning. However, this is very challenging because the target domain is unseen during training. Thus we propose a new setting some unlabelled data from the target domain is provided, which can bridge the gap between the source domain and the target domain. A benchmark for this setting is constructed using DomainNet \cite{peng2018oment}. We come up with a self-supervised learning meth…
When data is unlabelled and the target task is not known a priori, divergent search offers a strategy for learning a wide range of skills. Having such a repertoire allows a system to adapt to new, unforeseen tasks. Unlabelled image data is plentiful, but it is not always known which features will be required for downstream tasks. We propose a method for divergent search in the few-shot image classification setting and evaluate with Omniglot and Mini-ImageNet. This high-dimensional behavior space includes all possible ways of partitioning the data. To manage divergent search in this space, we r…
Università di Trento（特伦托大学）；Università di Modena e Reggio Emilia（摩德纳雷焦艾米利亚大学）
Future deep learning systems call for techniques that can deal with the evolving nature of temporal data and scarcity of annotations when new problems occur. As a step towards this goal, we present FUSION (Few-shot UnSupervIsed cONtinual learning), a learning strategy that enables a neural network to learn quickly and continually on streams of unlabelled data and unbalanced tasks. The objective is to maximise the knowledge extracted from the unlabelled data stream (unsupervised), favor the forward transfer of previously learnt tasks and features (continual) and exploit as much as possible the …
Transductive Few-Shot learning has gained increased attention nowadays considering the cost of data annotations along with the increased accuracy provided by unlabelled samples in the domain of few shot. Especially in Few-Shot Classification (FSC), recent works explore the feature distributions aiming at maximizing likelihoods or posteriors with respect to the unknown parameters. Following this vein, and considering the parallel between FSC and clustering, we seek for better taking into account the uncertainty in estimation due to lack of data, as well as better statistical properties of the c…
Few-shot classification is a challenging problem due to the uncertainty caused by using few labelled samples. In the past few years, many methods have been proposed with the common aim of transferring knowledge acquired on a previously solved task, what is often achieved by using a pretrained feature extractor. Following this vein, in this paper we propose a novel transfer-based method which aims at processing the feature vectors so that they become closer to Gaussian-like distributions, resulting in increased accuracy. In the case of transductive few-shot learning where unlabelled test sample…
Technical University of Munich（慕尼黑工业大学）；Helmholtz AI, Helmholtz Center Munich（亥姆霍兹人工智能，亥姆霍兹慕尼黑中心）；King Abdullah University of Science and Technology (KAUST)（阿卜杜拉国王科技大学）
Recent progress has shown that few-shot learning can be improved with access to unlabelled data, known as semi-supervised few-shot learning(SS-FSL). We introduce an SS-FSL approach, dubbed as Prototypical Random Walk Networks(PRWN), built on top of Prototypical Networks (PN). We develop a random walk semi-supervised loss that enables the network to learn representations that are compact and well-separated. Our work is related to the very recent development of graph-based approaches for few-shot learning. However, we show that compact and well-separated class representations can be achieved by …
Most visual recognition methods implicitly assume the data distribution remains unchanged from training to testing. However, in practice domain shift often exists, where real-world factors such as lighting and sensor type change between train and test, and classifiers do not generalise from source to target domains. It is impractical to train separate models for all possible situations because collecting and labelling the data is expensive. Domain adaptation algorithms aim to ameliorate domain shift, allowing a model trained on a source to perform well on a different target domain. However, ev…
In this paper, we aim to tackle the one-shot person re-identification problem where only one image is labelled for each person, while other images are unlabelled. This task is challenging due to the lack of sufficient labelled training data. To tackle this problem, we propose to iteratively guess pseudo labels for the unlabeled image samples, which are later used to update the re-identification model together with the labelled samples. A new sampling mechanism is designed to select unlabeled samples to pseudo labelled samples based on the distance matrix, and to form a training triplet batch i…
Enhancing Few-Shot Learning with Integrated Data and GAN Model Approaches
Columbia University（哥伦比亚大学）；University of Michigan（密歇根大学）；Tulane University（杜兰大学）；Northeastern University（东北大学）；Dartmouth College（达特茅斯学院）；Shanghai Jiao Tong University（上海交通大学）
Enhancing Unsupervised Graph Few-shot Learning via Set Functions and Optimal Transport
College of Computer Science and Technology, Jilin University（吉林大学计算机科学与技术学院）；Department of Information Engineering and Computer Science, University of Trento（特伦托大学信息工程与计算机科学系）
Prompt as Free Lunch: Enhancing Diversity in Source-Free Cross-domain Few-shot Learning through Semantic-Guided Prompting
College of Computer and Data Science, Fuzhou University（福州大学计算机与数据科学学院）；Zhejiang University of Technology（浙江工业大学）；Hangzhou International Innovation Institute, Beihang University（北京航空航天大学杭州国际创新研究院）；INSAIT, Sofia University（保加利亚索菲亚大学INSAIT研究院）
Self-Supervised and Few-Shot Learning for Robust Bioaerosol Monitoring
Lucerne University of Applied Sciences and Arts（卢塞恩应用科学与艺术大学）；Federal Office of Meteorology and Climatology MeteoSwiss（瑞士联邦气象与气候学办公室（瑞士气象局））；École Polytechnique Fédérale de Lausanne（洛桑联邦理工学院）；University of Basel（巴塞尔大学）；Swisens AG（斯维森斯公司）
AA-CLIP: Enhancing Zero-shot Anomaly Detection via Anomaly-Aware CLIP
School of Biomedical Engineering, Division of Life Sciences and Medicine, USTC（中国科学技术大学生命科学与医学部生物医学工程学院）；MIRACLE Center, Suzhou Institute for Advance Research, USTC（中国科学技术大学苏州高等研究院MIRACLE中心）；Key Laboratory of Intelligent Information Processing of CAS, ICT, CAS（中国科学院计算技术研究所中国科学院智能信息处理重点实验室）；State Key Laboratory of Precision and Intelligent Chemistry, USTC（中国科学技术大学精准与智能化学国家重点实验室）；Stanford University（斯坦福大学）
Enhancing Zero-Shot Anomaly Detection: CLIP-SAM Collaboration with Cascaded Prompts
School of Artificial Intelligence, Anhui University, Hefei, China（人工智能学院，安徽大学，合肥，中国）；Anhui Provincial Key Laboratory of Security Artificial Intelligence, Anhui University（安徽省安全人工智能重点实验室，安徽大学）；Anhui Provincial Engineering Research Center for Unmanned System（安徽省无人系统与智能技术工程研究中心）
本文提出一种用于工业异常检测的两阶段零样本异常分割框架，协同利用 CLIP 的异常定位与 SAM 的边界感知能力，通过 PPG 和 CPS 模块优化点提示与分割精度，在 Visa 等数据集上取得 SOTA 表现。
Adaptive Clipping for Privacy-Preserving Few-Shot Learning: Enhancing Generalization with Limited Data
Deakin University（迪肯大学）；Data61, CSIRO（联邦科学与工业研究组织Data61研究院）；The University of Alabama in Huntsville（阿拉巴马大学亨茨维尔分校）；University of New South Wales (UNSW)（新南威尔士大学）
We study the problem of learning to estimate the 3D object pose from a few labelled examples and a collection of unlabelled data. Our main contribution is a learning framework, neural view synthesis and matching, that can transfer the 3D pose annotation from the labelled to unlabelled images reliably, despite unseen 3D views and nuisance variations such as the object shape, texture, illumination or scene context. In our approach, objects are represented as 3D cuboid meshes composed of feature vectors at each mesh vertex. The model is initialized from a few labelled images and is subsequently u…
Xiamen University（厦门大学）；Tencent（腾讯）；Xi’an Jiaotong University（西安交通大学）；Macau University of Science and Technology（澳门科技大学）
We introduce a one-shot segmentation method to alleviate the burden of manual annotation for medical images. The main idea is to treat one-shot segmentation as a classical atlas-based segmentation problem, where voxel-wise correspondence from the atlas to the unlabelled data is learned. Subsequently, segmentation label of the atlas can be transferred to the unlabelled data with the learned correspondence. However, since ground truth correspondence between images is usually unavailable, the learning system must be well-supervised to avoid mode collapse and convergence failure. To overcome this …
Carnegie Mellon University（卡内基梅隆大学）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Few-shot classification aims at classifying categories of a novel task by learning from just a few (typically, 1 to 5) labelled examples. An effective approach to few-shot classification involves a prior model trained on a large-sample base domain, which is then finetuned over the novel few-shot task to yield generalizable representations. However, task-specific finetuning is prone to overfitting due to the lack of enough training examples. To alleviate this issue, we propose a new finetuning approach based on contrastive learning that reuses unlabelled examples from the base domain in the for…
Universität Ulm（乌尔姆大学）；Mercedes-Benz AG（梅赛德斯-奔驰股份公司）；Ulm University Medical Center（乌尔姆大学医学中心）；Centre for Vision, Speech and Signal Processing, University of Surrey（萨里大学视觉、语音与信号处理中心）；Friedrich-Alexander-Universität Erlangen-Nürnberg（埃尔朗根-纽伦堡弗里德里希-亚历山大大学）
In microscopy image cell segmentation, it is common to train a deep neural network on source data, containing different types of microscopy images, and then fine-tune it using a support set comprising a few randomly selected and annotated training target images. In this paper, we argue that the random selection of unlabelled training target images to be annotated and included in the support set may not enable an effective fine-tuning process, so we propose a new approach to optimise this image selection process. Our approach involves a new scoring function to find informative unlabelled target…
University of Science and Technology of China（中国科学技术大学）；Microsoft Research Asia（微软亚洲研究院）；Microsoft Research（微软研究院）；Hangzhou Dianzi University（杭州电子科技大学）
In this paper, we introduce the new ideas of augmenting Convolutional Neural Networks (CNNs) with Memory and learning to learn the network parameters for the unlabelled images on the fly in one-shot learning. Specifically, we present Memory Matching Networks (MM-Net) --- a novel deep architecture that explores the training procedure, following the philosophy that training and test conditions must match. Technically, MM-Net writes the features of a set of labelled images (support set) into memory and reads from memory when performing inference to holistically leverage the knowledge in the set. …
Universität Ulm（乌尔姆大学）；Ulm University Medical Center（乌尔姆大学医学中心）；The University of Adelaide（阿德莱德大学）；Otto von Guericke University（奥托·冯·格里克大学）
Deep neural networks currently deliver promising results for microscopy image cell segmentation, but they require large-scale labelled databases, which is a costly and time-consuming process. In this work, we relax the labelling requirement by combining self-supervised with semi-supervised learning. We propose the prediction of edge-based maps for self-supervising the training of the unlabelled images, which is combined with the supervised training of a small number of labelled images for learning the segmentation task. In our experiments, we evaluate on a few-shot microscopy image cell segmen…
Labeling large image datasets with attributes such as facial age or object type is tedious and sometimes infeasible. Supervised machine learning methods provide a highly accurate solution, but require manual labels which are often unavailable. Zero-shot models (e.g., CLIP) do not require manual labels but are not as accurate as supervised ones, particularly when the attribute is numeric. We propose a new approach, CLIPPR (CLIP with Priors), which adapts zero-shot models for regression and classification on unlabelled datasets. Our method does not use any annotated images. Instead, we assume a …
School of Electronic Engineering, Xidian University（西安电子科技大学电子工程学院）；College of Intelligence and Computing, Tianjin University（天津大学智能与计算学部）；Peng Cheng Laboratory（鹏城实验室）；University of Technology Sydney（悉尼科技大学）
Few-shot object detection (FSOD) aims to strengthen the performance of novel object detection with few labeled samples. To alleviate the constraint of few samples, enhancing the generalization ability of learned features for novel objects plays a key role. Thus, the feature learning process of FSOD should focus more on intrinsical object characteristics, which are invariant under different visual changes and therefore are helpful for feature generalization. Unlike previous attempts of the meta-learning paradigm, in this paper, we explore how to enhance object features with intrinsical characte…
Autodesk AI Lab（欧特克人工智能实验室）；Autodesk Research（欧特克研究院）
Generating shapes using natural language can enable new ways of imagining and creating the things around us. While significant recent progress has been made in text-to-image generation, text-to-shape generation remains a challenging problem due to the unavailability of paired text and shape data at a large scale. We present a simple yet effective method for zero-shot text-to-shape generation that circumvents such data scarcity. Our proposed method, named CLIP-Forge, is based on a two-stage training process, which only depends on an unlabelled shape dataset and a pre-trained image-text network …
National University of Defense Technology（国防科技大学）；Chinese Academy of Sciences（中国科学院）
Neural architecture search (NAS) has made tremendous progress in the automatic design of effective neural network structures but suffers from a heavy computational burden. One-shot NAS significantly alleviates the burden through weight sharing and improves computational efficiency. Zero-shot NAS further reduces the cost by predicting the performance of the network from its initial state, which conducts no training. Both methods aim to distinguish between "good" and "bad" architectures, i.e., ranking consistency of predicted and true performance. In this paper, we propose Ranking Distillation o…
We consider the task of multimodal one-shot speech-image matching. An agent is shown a picture along with a spoken word describing the object in the picture, e.g. cookie, broccoli and ice-cream. After observing one paired speech-image example per class, it is shown a new set of unseen pictures, and asked to pick the "ice-cream". Previous work attempted to tackle this problem using transfer learning: supervised models are trained on labelled background data not containing any of the one-shot classes. Here we compare transfer learning to unsupervised models trained on unlabelled in-domain data. …
The goal of this paper is to bypass the need for labelled examples in few-shot video understanding at run time. While proven effective, in many practical video settings even labelling a few examples appears unrealistic. This is especially true as the level of details in spatio-temporal video understanding and with it, the complexity of annotations continues to increase. Rather than performing few-shot learning with a human oracle to provide a few densely labelled support videos, we propose to automatically learn to find appropriate support videos given a query. We call this self-shot learning …
University of Science and Technology of China（中国科学技术大学）；The University of Manchester（曼彻斯特大学）
It is well-known that zero-shot learning (ZSL) can suffer severely from the problem of domain shift, where the true and learned data distributions for the unseen classes do not match. Although transductive ZSL (TZSL) attempts to improve this by allowing the use of unlabelled examples from the unseen classes, there is still a high level of distribution shift. We propose a novel TZSL model (named as Bi-VAEGAN), which largely improves the shift by a strengthened distribution alignment between the visual and auxiliary spaces. The key proposal of the model design includes (1) a bi-directional distr…
Aiming at recognizing the samples from novel categories with few reference samples, few-shot learning (FSL) is a challenging problem. We found that the existing works often build their few-shot model based on the image-level feature by mixing all local-level features, which leads to the discriminative location bias and information loss in local details. To tackle the problem, this paper returns the perspective to the local-level feature and proposes a series of local-level strategies. Specifically, we present (a) a local-agnostic training strategy to avoid the discriminative location bias betw…
Tsinghua University（清华大学）；Tencent Youtu Lab（腾讯优图实验室）；University College London（伦敦大学学院）
Few-shot semantic segmentation aims to segment novel-class objects in a query image with only a few annotated examples in support images. Most of advanced solutions exploit a metric learning framework that performs segmentation through matching each pixel to a learned foreground prototype. However, this framework suffers from biased classification due to incomplete construction of sample pairs with the foreground prototype only. To address this issue, in this paper, we introduce a complementary self-contrastive task into few-shot semantic segmentation. Our new model is able to associate the pi…
Enhancing Zero-shot Personalized Image Aesthetics Assessment with Profile-aware Multimodal LLM
Mashang Consumer Finance Co., Ltd.（马莎消费金融有限公司）；Xi'an Jiaotong-Liverpool University（西安交通大学利物浦大学）；Beijing University of Posts and Telecommunications（北京邮电大学）
SCOPE: Scene-Contextualized Incremental Few-Shot 3D Segmentation
University of Surrey, UK（英国萨里大学）；University of Wollongong, Australia（澳大利亚沃拉彭大学）；University of Copenhagen, Denmark（丹麦哥本哈根大学）；University of Exeter, UK（英国埃克塞特大学）；Singapore University of Technology and Design, Singapore（新加坡科技与设计大学）
fine-CLIP: Enhancing Zero-Shot Fine-Grained Surgical Action Recognition with Vision-Language Models
University of Strasbourg（斯特拉斯堡大学）；CNRS（法国国家科学研究中心）；INSERM（法国国家健康与医学研究院）；ICube（ICube实验室）；IHU Strasbourg（斯特拉斯堡大学医院研究所）；University Hospital of Strasbourg（斯特拉斯堡大学医院）
University of Oxford（牛津大学）；Shanghai Jiao Tong University（上海交通大学）
The objective of this paper is few-shot object detection (FSOD) -- the task of expanding an object detector for a new category given only a few instances for training. We introduce a simple pseudo-labelling method to source high-quality pseudo-annotations from the training set, for each new category, vastly increasing the number of training instances and reducing class imbalance; our method finds previously unlabelled instances. Naïvely training with model predictions yields sub-optimal performance; we present two novel methods to improve the precision of the pseudo-labelling process: first, w…
