Few-shot visual recognition refers to recognize novel visual concepts from a few labeled instances. Many few-shot visual recognition methods adopt the metric-based meta-learning paradigm by comparing the query representation with class representations to predict the category of query instance. However, current metric-based methods generally treat all instances equally and consequently often obtain biased class representation, considering not all instances are equally significant when summarizing the instance-level representations for the class-level representation. For example, some instances may contain unrepresentative information, such as too much background and information of unrelated concepts, which skew the results. To address the above issues, we propose a novel metric-based meta-learning framework termed instance-adaptive class representation learning network (ICRL-Net) for few-shot visual recognition. Specifically, we develop an adaptive instance revaluing network with the capability to address the biased representation issue when generating the class representation, by learning and assigning adaptive weights for different instances according to their relative significance in the support set of corresponding class. Additionally, we design an improved bilinear instance representation and incorporate two novel structural losses, i.e., intra-class instance clustering loss and inter-class representation distinguishing loss, to further regulate the instance revaluation process and refine the class representation. We conduct extensive experiments on four commonly adopted few-shot benchmarks: miniImageNet, tieredImageNet, CIFAR-FS, and FC100 datasets. The experimental results compared with the state-of-the-art approaches demonstrate the superiority of our ICRL-Net.
Not all tokens contribute equally to diffusion learning
School of Mechanical Engineering, Guizhou University, Guizhou, China(贵州大学机械工程学院);School of Information Science and Technology, Hangzhou Normal University, Zhejiang, China(杭州师范大学信息科学与技术学院)
Not All Tasks Quantize Equally: Fisher-Guided Quantization for Visual Geometry Transformer
Department of Electronic and Computer Engineering, HKUST(香港科技大学电子与计算机工程系);State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences(中国科学院人工智能安全国家重点实验室);University of Chinese Academy of Sciences(中国科学院大学);School of Data Science and Engineering, South China Normal University(华南师范大学数据科学与工程学院)
Reliable Shot Identification for Complex Event Detection via Visual-Semantic Embedding
School of Computer Science and Technology, Xi’an Jiaotong University(西安交通大学计算机科学与技术学院);School of Computing Technologies, RMIT University(皇家墨尔本理工大学计算技术学院);Nanjing University of Science and Technology(南京理工大学)
Image Deformation Meta-Networks for One-Shot Learning
Fudan University(复旦大学);Jilian Technology Group (Video++)(极链科技集团(Video++));Carnegie Mellon University(卡内基梅隆大学);Tencent AI Lab(腾讯人工智能实验室)
即使图像经历各种变形并丢失某些信息,人类也可以鲁棒地学习新的视觉概念。模仿相同的行为并合成新概念的变形实例可以帮助视觉识别系统执行更好的一次性学习,即,从一个或几个例子中学习概念。我们的关键见解是,虽然变形的图像可能在视觉上并不真实,但它们仍然保持着关键的语义信息,并对制定分类器决策边界做出了重大贡献。受元学习最新进展的启发,我们将元学习器与图像变形子网络相结合,生成额外的训练样本,并以端到端的方式优化两个模型。变形子网络通过融合一对图像来学习变形图像--保持视觉内容的探测图像和多样化变形的图库图像。我们展示了广泛使用的一次性学习基准测试(miniImageNet和ImageNet 1 K Challenge数据集)的结果,这些测试结果明显优于最先进的方法。代码可在https://github.com/tankche1/IDeMe-Net上获得。
MICM: Rethinking Unsupervised Pretraining for Enhanced Few-shot Learning
School of Computer Science and Technology, Huazhong University of Science and Technology(华中科技大学计算机科学与技术学院);School of Computer Science, Peking University(北京大学计算机学院)
The Chinese University of Hong Kong(香港中文大学);University of California, Los Angeles(加利福尼亚大学洛杉矶分校);Huawei Noah’s Ark Lab(华为诺亚方舟实验室);National University of Singapore(新加坡国立大学)
CD^2: Constrained Dataset Distillation for Few-Shot Class-Incremental Learning
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络安全学院);Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州))
Matching-Based Few-Shot Semantic Segmentation Models Are Interpretable by Design
Department of Computer Science, University of Bari Aldo Moro(巴里大学计算机科学系);Jheronimus Academy of Data Science (JADS)(赫尔蒙尼数据科学学院);Eindhoven University of Technology (TU/e)(埃因霍温理工大学)
AutoCLIP: Auto-tuning Zero-Shot Classifiers for Vision-Language Models
Bosch Center for Artificial Intelligence, Robert Bosch GmbH(博世人工智能中心,罗伯特·博世公司);Bosch Center for Artificial Intelligence, Robert Bosch LLC(博世人工智能中心,罗伯特·博世有限责任公司)
École Polytechnique de Montréal(蒙特利尔综合理工学院);McGill University(麦吉尔大学)
生成对抗网络(GAN)拥有令人印象深刻的生成逼真图像的能力。然而,与深度学习领域的许多领域一样,它们需要大量的数据才能产生结果,从而限制了它们在产生新颖性方面的有用性。同样,元学习的最新进展为许多少样本学习应用打开了大门。在目前的工作中,我们提出了Few-Shot Image Generation using Reptile(FIGR),一个使用Reptile进行元训练的GAN。我们的模型成功地在MNIST和Omniglot上生成了新的图像,其中只有4张来自看不见的类的图像。我们进一步贡献了FIGR-8,这是一个用于Few-Shot图像生成的新数据集,其中包含1,548,944个图标,分为18,409个类别。在FIGR-8上进行训练,初步结果表明,我们的模型可以从以前看不见的图像类别中的8个样本和这8个图像中的10个训练步骤中概括出更高级的概念(如“鸟”和“刀”)。这项工作展示了训练GAN用于Few-Shot图像生成的潜力,并旨在为该领域的未来工作设定新的基准。
Few-Shot Learning with Adaptive Weight Masking in Conditional GANs
Tulane University(杜兰大学);Northeastern University(东北大学);Washington University in St. Louis(华盛顿大学圣路易斯分校);New York University(纽约大学);Johns Hopkins University(约翰斯·霍普金斯大学)
IPFormer-VideoLLM: Enhancing Multi-modal Video Understanding for Multi-shot Scenes
School of AIA, Huazhong University of Science and Technology(华中科技大学人工智能学院);Deepeleph Intelligent Technology(DeepEleph智能技术);JD Explore Academy(JD探索学院);Department of Electronic Engineering, Tsinghua University(清华大学电子工程系)
Learning to Self-Train for Semi-Supervised Few-Shot Classification
Shanghai Jiao Tong University(上海交通大学);Singapore Management University(新加坡管理大学);Tianjin University(天津大学);Alibaba Group(阿里巴巴集团);National University of Singapore(新加坡国立大学);Max Planck Institute for Informatics(马克斯·普朗克信息学研究所);Saarland Informatics Campus(萨尔兰信息学园区)