Few-shot action recognition aims to enable models to quickly learn new action categories from limited labeled samples, addressing the challenge of data scarcity in real-world applications. Current research primarily addresses three core challenges: (1) temporal modeling, where models are prone to interference from irrelevant static background information and struggle to capture the essence of dynamic action features; (2) visual similarity, where categories with subtle visual differences are difficult to distinguish; and (3) the modality gap between visual-textual support prototypes and visual-only queries, which complicates alignment within a shared embedding space. To address these challenges, this paper proposes a CLIP-SPM framework, which includes three components: (1) the Hierarchical Synergistic Motion Refinement (HSMR) module, which aligns deep and shallow motion features to improve temporal modeling by reducing static background interference; (2) the Semantic Prototype Modulation (SPM) strategy, which generates query-relevant text prompts to bridge the modality gap and integrates them with visual features, enhancing the discriminability between similar actions; and (3) the Prototype-Anchor Dual Modulation (PADM) method, which refines support prototypes and aligns query features with a global semantic anchor, improving consistency across support and query samples. Comprehensive experiments across standard benchmarks, including Kinetics, SSv2-Full, SSv2-Small, UCF101, and HMDB51, demonstrate that our CLIP-SPM achieves competitive performance under 1-shot, 3-shot, and 5-shot settings. Extensive ablation studies and visual analyses further validate the effectiveness of each component and its contributions to addressing the core challenges. The source code and models are publicly available at GitHub.
University of Sydney（悉尼大学）；Sydney Institute for Robotics and Intelligent Systems（悉尼机器人与智能系统研究院）
Distinguishing visually similar objects like forged/authentic bills and healthy/unhealthy plants is beyond the capabilities of even the most sophisticated classifiers. We propose the use of multiplexed illumination to extend the range of objects that can be successfully classified. We construct a compact RGB-IR light stage that images samples under different combinations of illuminant position and colour. We then develop a methodology for selecting illumination patterns and training a classifier using the resulting imagery. We use the light stage to model and synthetically relight training sam…
Lacking the ability to sense ambient environments effectively, blind and visually impaired people (BVIP) face difficulty in walking outdoors, especially in urban areas. Therefore, tools for assisting BVIP are of great importance. In this paper, we propose a novel "flying guide dog" prototype for BVIP assistance using drone and street view semantic segmentation. Based on the walkable areas extracted from the segmentation prediction, the drone can adjust its movement automatically and thus lead the user to walk along the walkable path. By recognizing the color of pedestrian traffic lights, our p…
Few-shot Unknown Class Discovery of Hyperspectral Images with Prototype Learning and Clustering
School of Computer and Information Engineering, Henan Key Laboratory of Big Data Analysis and Processing, Henan Engineering Laboratory of Spatial Information Processing and Henan Industrial Technology Academy of Spatio-Temporal Big Data, Henan University（计算机与信息工程学院、河南省大数据分析与处理重点实验室、河南省空间信息处理工程实验室和河南省时空大数据产业技术研究院，河南大学）
University of Twente（特文特大学）；University of Duisburg-Essen（杜伊斯堡-埃森大学）
Image recognition with prototypes is considered an interpretable alternative for black box deep learning models. Classification depends on the extent to which a test image "looks like" a prototype. However, perceptual similarity for humans can be different from the similarity learned by the classification model. Hence, only visualising prototypes can be insufficient for a user to understand what a prototype exactly represents, and why the model considers a prototype and an image to be similar. We address this ambiguity and argue that prototypes should be explained. We improve interpretability …
Learning Adaptive and Visually Aligned Conditions for Generative Zero-Shot Learning
School of Computer Science and Engineering, Southeast University, Nanjing, China（东南大学计算机科学与工程学院）；Qilu University of Technology (Shandong Academy of Sciences), Jinan, China（齐鲁工业大学（山东省科学院））
Proximity-CLIP: Text-Guided Semantic Proximity Learning for Zero-Shot Anomaly Detection
School of Software Engineering, Xi’an Jiaotong University（西安交通大学软件学院）；Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University（西安交通大学人工智能与机器人研究所）
Semantically Guided Dynamic Visual Prototype Refinement for Compositional Zero-Shot Learning
National Key Laboratory of Radar Signal Processing, Xidian University（雷达信号处理国家级重点实验室，西安电子科技大学）；Research Institute of Systems Engineering, Academy of Military Science（系统工程研究所，军事科学院）
Learning Multi-Modal Prototypes for Cross-Domain Few-Shot Object Detection
University of Chinese Academy of Sciences（中国科学院大学）；The Hong Kong Polytechnic University（香港理工大学）；Zhejiang University（浙江大学）；The University of Southern Queensland（昆士兰大学）
A*STAR（新加坡科技研究局）；Nanyang Technological University（南洋理工大学）；Jiangxi Finance and Economics University（江西财经大学）
Current methods for few-shot segmentation (FSSeg) have mainly focused on improving the performance of novel classes while neglecting the performance of base classes. To overcome this limitation, the task of generalized few-shot semantic segmentation (GFSSeg) has been introduced, aiming to predict segmentation masks for both base and novel classes. However, the current prototype-based methods do not explicitly consider the relationship between base and novel classes when updating prototypes, leading to a limited performance in identifying true categories. To address this challenge, we propose a…
COMBINER: Composed Image Retrieval Guided by Attribute-based Neighbor Relations
School of Software, Shandong University（山东大学软件学院）；School of Computer Science and Technology, Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳）计算机科学与技术学院）；Department of Data Science, City University of Hong Kong（香港城市大学数据科学系）；Department of Computer Science and Engineering, Southern University of Science and Technology（南方科技大学计算机科学与工程系）
Context Disentangling and Prototype Inheriting for Robust Visual Grounding
Nanjing University of Science and Technology（南京理工大学）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；SenseTime Research（商汤科技研究院）
Institute of Computing Technology, CAS（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Beijing Academy of Artificial Intelligence（北京智源人工智能研究院）
Teaching machines to recognize a new category based on few training samples especially only one remains challenging owing to the incomprehensive understanding of the novel category caused by the lack of data. However, human can learn new classes quickly even given few samples since human can tell what discriminative features should be focused on about each category based on both the visual and semantic prior knowledge. To better utilize those prior knowledge, we propose the SEmantic Guided Attention (SEGA) mechanism where the semantic knowledge is used to guide the visual perception in a top-d…
Universal Domain Adaptation aims to transfer the knowledge between the datasets by handling two shifts: domain-shift and category-shift. The main challenge is correctly distinguishing the unknown target samples while adapting the distribution of known class knowledge from source to target. Most existing methods approach this problem by first training the target adapted known classifier and then relying on the single threshold to distinguish unknown target samples. However, this simple threshold-based approach prevents the model from considering the underlying complexities existing between the …
Explaining black-box Artificial Intelligence (AI) models is a cornerstone for trustworthy AI and a prerequisite for its use in safety critical applications such that AI models can reliably assist humans in critical decisions. However, instead of trying to explain our models post-hoc, we need models which are interpretable-by-design built on a reasoning process similar to humans that exploits meaningful high-level concepts such as shapes, texture or object parts. Learning such concepts is often hindered by its need for explicit specification and annotation up front. Instead, prototype-based lea…
Yonsei University（延世大学）；Korea Institute of Science and Technology（韩国科学技术研究院）
Salient object detection (SOD) is a task that involves identifying and segmenting the most visually prominent object in an image. Existing solutions can accomplish this use a multi-scale feature fusion mechanism to detect the global context of an image. However, as there is no consideration of the structures in the image nor the relations between distant pixels, conventional methods cannot deal with complex scenes effectively. In this paper, we propose an adaptive graph convolution module (AGCM) to overcome these limitations. Prototype features are initially extracted from the input image usin…
Technical University of Cluj-Napoca（克卢日-纳波卡技术大学）；Isaac Newton Group of Telescopes（艾萨克·牛顿望远镜组）；Instituto de Astrofisica de Canarias（加那利天体物理研究所）；Universidad de La Laguna（拉古纳大学）
Near Earth Asteroids (NEAs) are discovered daily, mainly by few major surveys, nevertheless many of them remain unobserved for years, even decades. Even so, there is room for new discoveries, including those submitted by smaller projects and amateur astronomers. Besides the well-known surveys that have their own automated system of asteroid detection, there are only a few software solutions designed to help amateurs and mini-surveys in NEAs discovery. Some of these obtain their results based on the blink method in which a set of reduced images are shown one after another and the astronomer has…
Instance-level Visual Active Tracking with Occlusion-Aware Planning
South China University of Technology（南方科技大学）；Pazhou Laboratory（Pazhou实验室）；Key Laboratory of Big Data and Intelligent Robot, Ministry of Education（教育部大数据与智能机器人重点实验室）；University of Chinese Academy of Sciences（中国科学院大学）
Background Semantics Matter: Cross-Task Feature Exchange Network for Clustered Infrared Small Target Detection
School of Computer Science and Engineering, Nanjing University of Science and Technology（南京理工大学计算机科学与工程学院）；Luoyang Institute of Electro-Optical Equipment under AVIC (Aviation Industry Corporation of China)（航空工业集团洛阳光电设备研究所）；College of Electronic Science and Technology, National University of Defense Technology（国防科技大学电子科学学院）；School of Computer Science, Guangdong University of Education（广东教育学院计算机科学学院）；VCIP, College of Computer Science, Nankai University（南开大学计算机科学学院）
Inter- and Intra-image Refinement for Few Shot Segmentation
Department of Biomedical Engineering, Peking University（北京大学生物医学工程系）；Institute of Medical Technology, Peking University Health Science Center, Peking University（北京大学医学部医学技术研究所）；National Biomedical Imaging Center, Peking University（北京大学国家生物医学成像中心）
SenseTime Group Limited（商汤科技集团有限公司）；The Chinese University of HongKong（香港中文大学）
Although person re-identification (ReID) has achieved significant improvement recently by enforcing part alignment, it is still a challenging task when it comes to distinguishing visually similar identities or identifying the occluded person. In these scenarios, magnifying details in each part features and selectively fusing them together may provide a feasible solution. In this work, we propose MagnifierNet, a triple-branch network which accurately mines details from whole to parts. Firstly, the holistic salient features are encoded by a global branch. Secondly, to enhance detailed representa…
Northwestern Polytechnical University（西北工业大学）；Inception Institute of Artificial Intelligence（起源人工智能研究院）；Terminus Group（特斯联集团）
Existing studies in few-shot semantic segmentation only focus on mining the target object information, however, often are hard to tell ambiguous regions, especially in non-target regions, which include background (BG) and Distracting Objects (DOs). To alleviate this problem, we propose a novel framework, namely Non-Target Region Eliminating (NTRE) network, to explicitly mine and eliminate BG and DO regions in the query. First, a BG Mining Module (BGMM) is proposed to extract the BG region via learning a general BG prototype. To this end, we design a BG loss to supervise the learning of BGMM on…
