We consider a novel formulation of the problem of Active Few-Shot Classification (AFSC) where the objective is to classify a small, initially unlabeled, dataset given a very restrained labeling budget. This problem can be seen as a rival paradigm to classical Transductive Few-Shot Classification (TFSC), as both these approaches are applicable in similar conditions. We first propose a methodology that combines statistical inference, and an original two-tier active learning strategy that fits well into this framework. We then adapt several standard vision benchmarks from the field of TFSC. Our experiments show the potential benefits of AFSC can be substantial, with gains in average weighted accuracy of up to 10% compared to state-of-the-art TFSC methods for the same labeling budget. We believe this new paradigm could lead to new developments and standards in data-scarce learning settings.
Knowledge-Data Dually Driven Paradigm for Accurate Landslide Susceptibility Prediction under Data-Scarce Conditions Using Geomorphic Priors and Tabular Foundation Model
School of Engineering and Technology, China University of Geosciences (Beijing)（工程与技术学院，中国地质大学（北京））；School of Geological Engineering and Geomatics, Chang'an University（地质工程与测绘学院，长安大学）
In the current COVID-19 pandemic situation, there is an urgent need to screen infected patients quickly and accurately. Using deep learning models trained on chest X-ray images can become an efficient method for screening COVID-19 patients in these situations. Deep learning approaches are already widely used in the medical community. However, they require a large amount of data to be accurate. The open-source community collectively has made efforts to collect and annotate the data, but it is not enough to train an accurate deep learning model. Few-shot learning is a sub-field of machine learni…
TEPI: Taxonomy-aware Embedding and Pseudo-Imaging for Scarcely-labeled Zero-shot Genome Classification
Auburn University（奥本大学）；Oklahoma State University（俄克拉荷马州立大学）；Oklahoma Animal Disease Diagnostics Laboratory, College of Veterinary Medicine, Oklahoma State University（俄克拉荷马州立大学兽医学院俄克拉荷马动物疾病诊断实验室）
Meteorology-Driven GPT4AP: A Multi-Task Forecasting LLM for Atmospheric Air Pollution in Data-Scarce Settings
ADAPT SFI Research Centre, School of Computer Science, Technological University Dublin（ADAPT SFI研究中心，计算机科学学院，都柏林理工大学）；ADAPT SFI Research Centre, School of Computer Science, University College Dublin（ADAPT SFI研究中心，计算机科学学院，都柏林大学学院）；School of Computer Science, Technological University Dublin（计算机科学学院，都柏林理工大学）
Arizona State University（亚利桑那州立大学）；University of Virginia（弗吉尼亚大学）
Few-shot node classification is tasked to provide accurate predictions for nodes from novel classes with only few representative labeled nodes. This problem has drawn tremendous attention for its projection to prevailing real-world applications, such as product categorization for newly added commodity categories on an E-commerce platform with scarce records or diagnoses for rare diseases on a patient similarity graph. To tackle such challenging label scarcity issues in the non-Euclidean graph domain, meta-learning has become a successful and predominant paradigm. More recently, inspired by the…
SAP SE（思爱普股份公司）；University of Trento（特伦托大学）；TU Berlin（柏林工业大学）
Since the advent of deep learning, neural networks have demonstrated remarkable results in many visual recognition tasks, constantly pushing the limits. However, the state-of-the-art approaches are largely unsuitable in scarce data regimes. To address this shortcoming, this paper proposes employing a 3D model, which is derived from training images. Such a model can then be used to hallucinate novel viewpoints and poses for the scarce samples of the few-shot learning scenario. A self-paced learning approach allows for the selection of a diverse set of high-quality images, which facilitates the …
Being able to learn from small amounts of data is a key characteristic of human intelligence, but exactly {\em how} small? In this paper, we introduce a novel experimental paradigm that allows us to examine classification in an extremely data-scarce setting, asking whether humans can learn more categories than they have exemplars (i.e., can humans do "less-than-one shot" learning?). An experiment conducted using this paradigm reveals that people are capable of learning in such settings, and provides several insights into underlying mechanisms. First, people can accurately infer and represent h…
SSMT: Few-Shot Traffic Forecasting with Single Source Meta-Transfer
Sungkyunkwan University（成均馆大学）；University of California, Riverside（加州大学河滨分校）；Center for Computational & Data Sciences, Independent University, Bangladesh（孟加拉国独立大学计算与数据科学中心）
Developing mechanisms that flexibly adapt dialog systems to unseen tasks and domains is a major challenge in dialog research. Neural models implicitly memorize task-specific dialog policies from the training data. We posit that this implicit memorization has precluded zero-shot transfer learning. To this end, we leverage the schema-guided paradigm, wherein the task-specific dialog policy is explicitly provided to the model. We introduce the Schema Attention Model (SAM) and improved schema representations for the STAR corpus. SAM obtains significant improvement in zero-shot settings, with a +22…
It is very common to face classification problems where the number of available labeled samples is small compared to their dimension. These conditions are likely to cause underdetermined settings, with high risk of overfitting. To improve the generalization ability of trained classifiers, common solutions include using priors about the data distribution. Among many options, data structure priors, often represented through graphs, are increasingly popular in the field. In this paper, we introduce a generic model where observed class signals are supposed to be deteriorated with two sources of no…
Few-Shot Bioactivity Prediction with Meta-Learning under Assay Heterogeneity
Helmholtz Munich（亥姆霍兹慕尼黑研究中心）；Technical University of Munich（慕尼黑工业大学）；Helmholtz AI（亥姆霍兹人工智能中心）；Munich Center for Machine Learning（慕尼黑机器学习中心）；University of Warsaw（华沙大学）；University of Technology Nuremberg（纽伦堡工业大学）；LMU Munich（慕尼黑大学）
Arizona State University（亚利桑那州立大学）；Texas A&M University（得克萨斯农工大学）
Inspired by the extensive success of deep learning, graph neural networks (GNNs) have been proposed to learn expressive node representations and demonstrated promising performance in various graph learning tasks. However, existing endeavors predominately focus on the conventional semi-supervised setting where relatively abundant gold-labeled nodes are provided. While it is often impractical due to the fact that data labeling is unbearably laborious and requires intensive domain knowledge, especially when considering the heterogeneity of graph-structured data. Under the few-shot semi-supervised…
From Few-Shot to Zero-Shot: Towards Generalist Graph Anomaly Detection
School of Information and Communication Technology, Griffith University, Australia（信息与通信技术学院，格里菲斯大学，澳大利亚）；School of Computer, Electronics and Information, Guangxi University, Nanning, China（计算机、电子与信息学院，广西大学，南宁，中国）；Department of Data Science and Artificial Intelligence, The Hong Kong Polytechnic University, Hong Kong（数据科学与人工智能系，香港理工大学，香港）；Department of Computer Science, University of Illinois at Chicago, Chicago, IL 60607-7053, USA（计算机科学系，伊利诺伊大学香槟分校，芝加哥，IL 60607-7053，美国）
Mind the Gap: Bridging Prior Shift in Realistic Few-Shot Crop-Type Classification
Technical University of Munich (TUM)（慕尼黑技术大学）；TUM School of Engineering and Design（慕尼黑技术大学工程与设计学院）；Department of Aerospace and Geodesy（航空航天与大地测量系）；Chair of Remote Sensing Technology（遥感技术教授团）；Munich Data Science Institute (MDSI)（慕尼黑数据科学研究所）；ELLIS Unit Jena（耶拿ELLIS单元）；University of Jena（耶拿大学）
Zero and Few Shot Load Forecasting with Large Language Models
School of Electrical Engineering, Southeast University（东南大学电气工程学院）；Wind Engineering and Renewable Energy Laboratory, Ecole Polytechnique Federale de Lausanne (EPFL)（瑞士联邦理工学院洛桑分校风能与可再生能源实验室）；College of Electrical Engineering and New Energy, China Three Gorges University（中国三峡大学电气工程与新能源学院）；Department of Electrical and Electronic Engineering, Imperial College London（伦敦帝国理工学院电子与电气工程系）；The Department of Automation, School of Automation and Intelligent Sensing, Shanghai Jiao Tong University（上海交通大学自动化与智能感知学院）；The Key Laboratory of System Control and Information Processing, Ministry of Education of China, Shanghai（中国教育部系统控制与信息处理重点实验室，上海）；State Key Laboratory of Submarine Geoscience, Shanghai（上海 submarine 地球科学国家重点实验室）；Institute of Energy Systems, Energy Efficiency and Energy Economic, TU Dortmund University（德意志图林根大学能源系统、能效与能源经济研究所）
Improving Open-world Continual Learning under the Constraints of Scarce Labeled Data
Southwestern University of Finance and Economics（西南财经大学）；Leiden University（莱顿大学）；JD Intelligent Cities Research（京东智能城市研究院）；Southwest Jiaotong University（西南交通大学）
本文研究开放世界少样本持续学习问题，提出集成 ITA、MOB 和 AKS 的 OFCL 框架，以缓解遗忘、过拟合、开放检测及未知向已知更新难题。
Fraunhofer IAIS（弗劳恩霍夫智能分析与信息系统研究所）；University of Bonn（波恩大学）；PWC GmbH WPG（普华永道会计师事务所（德国））
Natural language processing methods have several applications in automated auditing, including document or passage classification, information retrieval, and question answering. However, training such models requires a large amount of annotated data which is scarce in industrial settings. At the same time, techniques like zero-shot and unsupervised learning allow for application of models pre-trained using general domain data to unseen domains. In this work, we study the efficiency of unsupervised text matching using Sentence-Bert, a transformer-based model, by applying it to the semantic simi…
National Key Laboratory for Novel Software Technology,Nanjing University（南京大学计算机软件新技术国家重点实验室）
Few-shot classification algorithms can alleviate the data scarceness issue, which is vital in many real-world problems, by adopting models pre-trained from abundant data in other domains. However, the pre-training process was commonly unaware of the future adaptation to other concept classes. We disclose that a classically fully trained feature extractor can leave little embedding space for unseen classes, which keeps the model from well-fitting the new classes. In this work, we propose to use a novelty-prepared loss function, called self-compacting softmax loss (SSL), for few-shot classificat…
Benchmarking for Practice: Few-Shot Time-Series Crop-Type Classification on the EuroCropsML Dataset
Technical University of Munich (TUM)（慕尼黑技术大学）；TUM School of Engineering and Design（慕尼黑技术大学工程与设计学院）；Department of Aerospace and Geodesy（航空航天与大地测量系）；Chair of Remote Sensing Technology（遥感技术教授团）；Munich Data Science Institute (MDSI)（慕尼黑数据科学研究所）；ELLIS Unit Jena（耶纳ELLIS单元）；University of Jena（耶纳大学）；ETH Zurich（苏黎世联邦理工学院）；Department of Mathematics（数学系）；Zuse Institute Berlin（柏林Zuse研究所）
SAP SE（SAP公司）；Humboldt-Universität zu Berlin（柏林洪堡大学）
State-of-the-art deep learning algorithms yield remarkable results in many visual recognition tasks. However, they still fail to provide satisfactory results in scarce data regimes. To a certain extent this lack of data can be compensated by multimodal information. Missing information in one modality of a single data point (e.g. an image) can be made up for in another modality (e.g. a textual description). Therefore, we design a few-shot learning task that is multimodal during training (i.e. image and text) and single-modal during test time (i.e. image). In this regard, we propose a self-paced…
In-Context Guidance: Learning Inter-Task Synergies via Numerical Foundational Models for Few-Shot Multitask Optimization
Nanyang Technological University（南洋理工大学）；Agency for Science, Technology and Research（新加坡科技研究局）；Singapore Institute of Manufacturing Technology (SIMTech)（新加坡制造技术研究院）；Centre for Frontier AI Research, A*STAR（A*STAR前沿人工智能研究中心）
PM2: A New Prompting Multi-modal Model Paradigm for Few-shot Medical Image Classification
Dalian University of Technology（大连理工大学）；School of Computer Science and Technology（计算机科学与技术学院）；School of Computer Science and Engineering（计算机科学与工程学院）；Dalian Minzu University（大连民族大学）；School of Information and Communication Engineering（信息与通信工程学院）
Given the success with in-context learning of large pre-trained language models, we introduce in-context learning distillation to transfer in-context few-shot learning ability from large models to smaller models. We propose to combine in-context learning objectives with language modeling objectives to distill both the ability to read in-context examples and task knowledge to the smaller models. We perform in-context learning distillation under two different few-shot learning paradigms: Meta In-context Tuning (Meta-ICT) and Multitask In-context Tuning (Multitask-ICT). Multitask-ICT performs bet…
Samsung AI Center Cambridge（三星剑桥人工智能中心）；University of Edinburgh（爱丁堡大学）
Few-shot learning (FSL) is an important and topical problem in computer vision that has motivated extensive research into numerous methods spanning from sophisticated meta-learning methods to simple transfer learning baselines. We seek to push the limits of a simple-but-effective pipeline for more realistic and practical settings of few-shot image classification. To this end, we explore few-shot learning from the perspective of neural network architecture, as well as a three stage pipeline of network updates under different data supplies, where unsupervised external data is considered for pre-…
Brown University（布朗大学）；Isfahan University of Tech.（伊斯法罕理工大学）
In many practical few-shot learning problems, even though labeled examples are scarce, there are abundant auxiliary datasets that potentially contain useful information. We propose the problem of extended few-shot learning to study these scenarios. We then introduce a framework to address the challenges of efficiently selecting and effectively using auxiliary data in few-shot image classification. Given a large auxiliary dataset and a notion of semantic similarity among classes, we automatically select pseudo shots, which are labeled examples from other classes related to the target task. We s…
University of Chinese Academy of Sciences（中国科学院大学）；Beijing Unisound Information Technology Co., Ltd（北京云知声信息技术有限公司）
The training of deep-learning-based text classification models relies heavily on a huge amount of annotation data, which is difficult to obtain. When the labeled data is scarce, models tend to struggle to achieve satisfactory performance. However, human beings can distinguish new categories very efficiently with few examples. This is mainly due to the fact that human beings can leverage knowledge obtained from relevant tasks. Inspired by human intelligence, we propose to introduce external knowledge into few-shot learning to imitate human knowledge. A novel parameter generator network is inves…
Few-shot learning (FSL) aims to recognize new objects with extremely limited training data for each category. Previous efforts are made by either leveraging meta-learning paradigm or novel principles in data augmentation to alleviate this extremely data-scarce problem. In contrast, this paper presents a simple statistical approach, dubbed Instance Credibility Inference (ICI) to exploit the distribution support of unlabeled instances for few-shot learning. Specifically, we first train a linear classifier with the labeled few-shot examples and use it to infer the pseudo-labels for the unlabeled …
ILLC, University of Amsterdam（阿姆斯特丹大学语言、逻辑与计算研究所（ILLC））；Facebook AI（Facebook人工智能（Facebook AI））；Dept. of Informatics, King’s College London（伦敦国王学院信息学系）
The success of deep learning methods hinges on the availability of large training datasets annotated for the task of interest. In contrast to human intelligence, these methods lack versatility and struggle to learn and adapt quickly to new tasks, where labeled data is scarce. Meta-learning aims to solve this problem by training a model on a large number of few-shot tasks, with an objective to learn new tasks quickly from a small number of examples. In this paper, we propose a meta-learning framework for few-shot word sense disambiguation (WSD), where the goal is to learn to disambiguate unseen…
In recent years, the majority of works on deep-learning-based image colorization have focused on how to make a good use of the enormous datasets currently available. What about when the data at disposal are scarce? The main objective of this work is to prove that a network can be trained and can provide excellent colorization results even without a large quantity of data. The adopted approach is a mixed one, which uses an adversarial method for the actual colorization, and a meta-learning technique to enhance the generator model. Also, a clusterization a-priori of the training dataset ensures …
Indian Institute of Technology Hyderabad（印度理工学院海得拉巴分校）
We propose to study the problem of few shot graph classification in graph neural networks (GNNs) to recognize unseen classes, given limited labeled graph examples. Despite several interesting GNN variants being proposed recently for node and graph classification tasks, when faced with scarce labeled examples in the few shot setting, these GNNs exhibit significant loss in classification performance. Here, we present an approach where a probability measure is assigned to each graph based on the spectrum of the graphs normalized Laplacian. This enables us to accordingly cluster the graph base lab…
University of Virginia（弗吉尼亚大学）；Arizona State University（亚利桑那州立大学）
Few-shot node classification aims at classifying nodes with limited labeled nodes as references. Recent few-shot node classification methods typically learn from classes with abundant labeled nodes (i.e., meta-training classes) and then generalize to classes with limited labeled nodes (i.e., meta-test classes). Nevertheless, on real-world graphs, it is usually difficult to obtain abundant labeled nodes for many classes. In practice, each meta-training class can only consist of several labeled nodes, known as the extremely weak supervision problem. In few-shot node classification, with extremel…
Deep neural network models have recently achieved state-of-the-art performance gains in a variety of natural language processing (NLP) tasks (Young, Hazarika, Poria, & Cambria, 2017). However, these gains rely on the availability of large amounts of annotated examples, without which state-of-the-art performance is rarely achievable. This is especially inconvenient for the many NLP fields where annotated examples are scarce, such as medical text. To improve NLP models in this situation, we evaluate five improvements on named entity recognition (NER) tasks when only ten annotated examples are av…
Differentiable architecture search (DAS) is a widely researched tool for the discovery of novel architectures, due to its promising results for image classification. The main benefit of DAS is the effectiveness achieved through the weight-sharing one-shot paradigm, which allows efficient architecture search. In this work, we investigate DAS in a systematic case study of inverse problems, which allows us to analyze these potential benefits in a controlled manner. We demonstrate that the success of DAS can be extended from image classification to signal reconstruction, in principle. However, our…
