Recent advancements in few-shot segmentation (FSS) have exploited pixel-by-pixel matching between query and support features, typically based on cross attention, which selectively activate query foreground (FG) features that correspond to the same-class support FG features. However, due to the large receptive fields in deep layers of the backbone, the extracted query and support FG features are inevitably mingled with background (BG) features, impeding the FG-FG matching in cross attention. Hence, the query FG features are fused with less support FG features, i.e., the support information is not well utilized. This paper presents a novel plug-in termed ambiguity elimination network (AENet), which can be plugged into any existing cross attention-based FSS methods. The main idea is to mine discriminative query FG regions to rectify the ambiguous FG features, increasing the proportion of FG information, so as to suppress the negative impacts of the doped BG features. In this way, the FG-FG matching is naturally enhanced. We plug AENet into three baselines CyCTR, SCCAN and HDMNet for evaluation, and their scores are improved by large margins, e.g., the 1-shot performance of SCCAN can be improved by 3.0%+ on both PASCAL-5$^i$ and COCO-20$^i$. The code is available at https://github.com/Sam1224/AENet.
School of Computer Science and Technology, Xidian University（西安电子科技大学计算机科学与技术学院）；Department of Mathematics, University of California, San Diego（加州大学圣地亚哥分校数学系）；School of Software Engineering, Xi'an Jiaotong University（西安交通大学软件工程学院）
Ambiguity-Guided Learnable Distribution Calibration for Semi-Supervised Few-Shot Class-Incremental Learning
New Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所模式识别新实验室）；School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University（上海交通大学电子信息与电气工程学院）；School of Electronics and Information Engineering, Suzhou University of Science and Technology（苏州科技大学电子与信息工程学院）；Jiangsu Industrial Intelligent Low Carbon Technology Engineering Center（江苏省工业智能低碳技术工程中心）
University of Science and Technology of China（中国科学技术大学）；National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所模式识别国家重点实验室）
Recent methods focus on learning a unified semantic-aligned visual representation to transfer knowledge between two domains, while ignoring the effect of semantic-free visual representation in alleviating the biased recognition problem. In this paper, we propose a novel Domain-aware Visual Bias Eliminating (DVBE) network that constructs two complementary visual representations, i.e., semantic-free and semantic-aligned, to treat seen and unseen domains separately. Specifically, we explore cross-attentive second-order visual statistics to compact the semantic-free representation, and design an a…
National Laboratory of Pattern Recognition, CASIA（中国科学院自动化研究所模式识别国家重点实验室）；USTC（中国科学技术大学）
Transductive Zero-shot learning (ZSL) targets to recognize the unseen categories by aligning the visual and semantic information in a joint embedding space. There exist four kinds of domain biases in Transductive ZSL, i.e., visual bias and semantic bias between two domains and two visual-semantic biases in respective seen and unseen domains, but existing work only focuses on the part of them, which leads to severe semantic ambiguity during the knowledge transfer. To solve the above problem, we propose a novel Attribute-Induced Bias Eliminating (AIBE) module for Transductive ZSL. Specifically, …
Northwestern Polytechnical University（西北工业大学）；Inception Institute of Artificial Intelligence（起源人工智能研究院）；Terminus Group（特斯联集团）
Existing studies in few-shot semantic segmentation only focus on mining the target object information, however, often are hard to tell ambiguous regions, especially in non-target regions, which include background (BG) and Distracting Objects (DOs). To alleviate this problem, we propose a novel framework, namely Non-Target Region Eliminating (NTRE) network, to explicitly mine and eliminate BG and DO regions in the query. First, a BG Mining Module (BGMM) is proposed to extract the BG region via learning a general BG prototype. To this end, we design a BG loss to supervise the learning of BGMM on…
Recovering multi-person 3D poses with absolute scales from a single RGB image is a challenging problem due to the inherent depth and scale ambiguity from a single view. Addressing this ambiguity requires to aggregate various cues over the entire image, such as body sizes, scene layouts, and inter-person relationships. However, most previous methods adopt a top-down scheme that first performs 2D pose detection and then regresses the 3D pose and scale for each detected person individually, ignoring global contextual cues. In this paper, we propose a novel system that first regresses a set of 2.5…
University of Chinese Academy of Sciences（中国科学院大学）
Few-shot segmentation is challenging because objects within the support and query images could significantly differ in appearance and pose. Using a single prototype acquired directly from the support image to segment the query image causes semantic ambiguity. In this paper, we propose prototype mixture models (PMMs), which correlate diverse image regions with multiple prototypes to enforce the prototype-based semantic representation. Estimated by an Expectation-Maximization algorithm, PMMs incorporate rich channel-wised and spatial semantics from limited support images. Utilized as representat…
Near Field iToF LIDAR Depth Improvement from Limited Number of Shots
Valeo Schalter und Sensoren GmbH（法雷奥开关与传感器有限公司）；Valeo Detection Systems GmbH（法雷奥检测系统有限公司）；Valeo Vision Systems（法雷奥视觉系统公司）；University of Limerick（利默里克大学）
Zero-Shot Automatic Annotation and Instance Segmentation using LLM-Generated Datasets: Eliminating Field Imaging and Manual Annotation for Deep Learning Model Development
DiffuSAM: Diffusion-Based Prompt-Free SAM2 for Few-Shot and Source-Free Medical Image Segmentation
School of Electrical Engineering, Tel Aviv University, Israel（特拉维夫大学电气工程学院）；School of Biomedical Engineering, Tel Aviv University, Israel（特拉维夫大学生物医学工程学院）
In recent works, utilizing a deep network trained on meta-training set serves as a strong baseline in few-shot learning. In this paper, we move forward to refine novel-class features by finetuning a trained deep network. Finetuning is designed to focus on reducing biases in novel-class feature distributions, which we define as two aspects: class-agnostic and class-specific biases. Class-agnostic bias is defined as the distribution shifting introduced by domain difference, which we propose Distribution Calibration Module(DCM) to reduce. DCM owes good property of eliminating domain difference an…
Don't Guess, Just Ask: Resolving Ambiguity in Referring Segmentation via Multi-turn Clarification
School of Computer Science and Engineering, Sun Yat-sen University（中山大学计算机科学与工程学院）；Guangdong Province Key Laboratory of Information Security Technology（广东省信息安全技术重点实验室）；Key Laboratory of Machine Intelligence and Advanced Computing, Ministry of Education（教育部机器智能与先进计算重点实验室）
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）
In autonomous driving, the novel objects and lack of annotations challenge the traditional 3D LiDAR semantic segmentation based on deep learning. Few-shot learning is a feasible way to solve these issues. However, currently few-shot semantic segmentation methods focus on camera data, and most of them only predict the novel classes without considering the base classes. This setting cannot be directly applied to autonomous driving due to safety concerns. Thus, we propose a few-shot 3D LiDAR semantic segmentation method that predicts both novel classes and base classes simultaneously. Our method …
Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China 2 School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China（1 中国科学院信息工程研究所，北京，中国 2 中国科学院大学网络空间安全学院，北京，中国）
One-shot talking head generation produces lip-sync talking heads based on arbitrary audio and one source face. To guarantee the naturalness and realness, recent methods propose to achieve free pose control instead of simply editing mouth areas. However, existing methods do not preserve accurate identity of source face when generating head motions. To solve the identity mismatch problem and achieve high-quality free pose control, we present One-shot Pose-controllable Talking head generation network (OPT). Specifically, the Audio Feature Disentanglement Module separates content features from aud…
Attentional Prototype Inference for Few-Shot Segmentation
Shandong University（山东大学）；Alibaba Group（阿里巴巴集团）；University of Amsterdam（阿姆斯特丹大学）；Inception Institute of Artificial Intelligence（起源人工智能研究院）；United Imaging Intelligence（联影智能）
Relevant Intrinsic Feature Enhancement Network for Few-Shot Semantic Segmentation
School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Alibaba group（阿里巴巴集团）
ZeroReg: Zero-Shot Point Cloud Registration with Foundation Models
University of Trento（特伦托大学）；Fondazione Bruno Kessler（布鲁诺·凯塞勒基金会）；East China University of Science and Technology（华东理工大学）；ETH Zurich（苏黎世联邦理工学院）；Shanghai AI Laboratory（上海人工智能实验室）
PANDORA: Pixel-wise Attention Dissolution and Latent Guidance for Zero-Shot Object Removal
University of Science, Ho Chi Minh City, Vietnam（越南胡志明市理科大学）；Vietnam National University, Ho Chi Minh City, Vietnam（越南胡志明市国家大学）；University of Dayton, Ohio, United States（美国俄亥俄州代顿大学）
Unleashing In-context Learning of Autoregressive Models for Few-shot Image Manipulation
GenAI, Meta（Meta GenAI部门）；University of Wisconsin–Madison（威斯康星大学麦迪逊分校）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Sungkyunkwan University（成均馆大学）；Georgia Institute of Technology（佐治亚理工学院）
The success of deep learning models is heavily tied to the use of massive amount of labeled data and excessively long training time. With the emergence of intelligent edge applications that use these models, the critical challenge is to obtain the same inference capability on a resource-constrained device while providing adaptability to cope with the dynamic changes in the data. We propose AgileNet, a novel lightweight dictionary-based few-shot learning methodology which provides reduced complexity deep neural network for efficient execution at the edge while enabling low-cost updates to captu…
RARE: Refine Any Registration of Pairwise Point Clouds via Zero-Shot Learning
College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics（南京航空航天大学计算机科学与技术学院）；Shenzhen Research Institute, Nanjing University of Aeronautics and Astronautics（南京航空航天大学深圳研究院）；School of Data Science, Lingnan University（岭南大学数据科学学院）
Fudan University（复旦大学）；Shanghai Key Lab of Intelligent Information Processing（上海市智能信息处理重点实验室）；The University of Hong Kong（香港大学）
This paper presents new hierarchically cascaded transformers that can improve data efficiency through attribute surrogates learning and spectral tokens pooling. Vision transformers have recently been thought of as a promising alternative to convolutional neural networks for visual recognition. But when there is no sufficient data, it gets stuck in overfitting and shows inferior performance. To improve data efficiency, we propose hierarchically cascaded transformers that exploit intrinsic image structures through spectral tokens pooling and optimize the learnable parameters through latent attri…
University of Tsukuba（筑波大学）；RIKEN Center for Advanced Intelligence Project（理化学研究所先进智能项目中心）
We investigate policy transfer using image-to-semantics translation to mitigate learning difficulties in vision-based robotics control agents. This problem assumes two environments: a simulator environment with semantics, that is, low-dimensional and essential information, as the state space, and a real-world environment with images as the state space. By learning mapping from images to semantics, we can transfer a policy, pre-trained in the simulator, to the real world, thereby eliminating real-world on-policy agent interactions to learn, which are costly and risky. In addition, using image-t…
South China University of Technology（华南理工大学）；University of Southampton（南安普顿大学）；University of Lincoln（林肯大学）
Zero-shot learning uses semantic attributes to connect the search space of unseen objects. In recent years, although the deep convolutional network brings powerful visual modeling capabilities to the ZSL task, its visual features have severe pattern inertia and lack of representation of semantic relationships, which leads to severe bias and ambiguity. In response to this, we propose the Graph-based Visual-Semantic Entanglement Network to conduct graph modeling of visual features, which is mapped to semantic attributes by using a knowledge graph, it contains several novel designs: 1. it establi…
