In many classification tasks, the set of target classes can be organized into a hierarchy. This structure induces a semantic distance between classes, and can be summarised under the form of a cost matrix, which defines a finite metric on the class set. In this paper, we propose to model the hierarchical class structure by integrating this metric in the supervision of a prototypical network. Our method relies on jointly learning a feature-extracting network and a set of class prototypes whose relative arrangement in the embedding space follows an hierarchical metric. We show that this approach allows for a consistent improvement of the error rate weighted by the cost matrix when compared to traditional methods and other prototype-based strategies. Furthermore, when the induced metric contains insight on the data structure, our method improves the overall precision as well. Experiments on four different public datasets - from agricultural time series classification to depth image semantic segmentation - validate our approach.
Semantic segmentation methods can not directly identify abnormal objects in images. Anomaly Segmentation algorithm from this realistic setting can distinguish between in-distribution objects and Out-Of-Distribution (OOD) objects and output the anomaly probability for pixels. In this paper, a Prototype Guided Anomaly segmentation Network (PGAN) is proposed to extract semantic prototypes for in-distribution training data from limited annotated images. In the model, prototypes are used to model the hierarchical category semantic information and distinguish OOD pixels. The proposed PGAN model incl…
SPENet: Self-guided Prototype Enhancement Network for Few-shot Medical Image Segmentation
School of Computer Science, Beijing University of Technology, Beijing, China（北京理工大学计算机科学学院）；Institute of Automation Chinese Academy of Sciences, Beijing, China（中国科学院自动化研究所）；Multimedia Department Xiaomi Inc, Beijing, China（小米公司多媒体部门）；Capital Medical University Affiliated Beijing Friendship Hospital, Beijing, China（首都医科大学附属北京友谊医院）
Samsung Research UK（三星英国研究院）；University of Padova（帕多瓦大学）
Federated Learning (FL) is a framework which enables distributed model training using a large corpus of decentralized training data. Existing methods aggregate models disregarding their internal representations, which are crucial for training models in vision tasks. System and statistical heterogeneity (e.g., highly imbalanced and non-i.i.d. data) further harm model training. To this end, we introduce a method, called FedProto, which computes client deviations using margins of prototypical representations learned on distributed data, and applies them to drive federated optimization via an atte…
DualProtoSeg: Simple and Efficient Design with Text- and Image-Guided Prototype Learning for Weakly Supervised Histopathology Image Segmentation
University of Houston（德克萨斯大学休斯顿分校）；Ho Chi Minh City University of Technology（胡志明市技术大学）；University of Information Technology（信息技术大学）；University of Technology Sydney（悉尼技术大学）；Vin University（文大学）；Department of Computer Science, Emory University（埃默里大学计算机科学系）；College of Medical Science and Technology, Taipei Medical University（台北医学院医学科学与技术学院）；Montefiore Medical Center, Albert Einstein College of Medicine（蒙特福医院，阿尔伯特·爱因斯坦医学院）；School of Computer Science, Carnegie Mellon University（卡内基梅隆大学计算机科学学院）
Late-decoupled 3D Hierarchical Semantic Segmentation with Semantic Prototype Discrimination based Bi-branch Supervision
School of Computing and AI, Southwest Jiaotong University, Chengdu, China（计算机与人工智能学院，西南交通大学，成都，中国）；Singapore University of Technology and Design, Singapore（新加坡科技设计大学，新加坡）
Referring image segmentation aims to segment the image region of interest according to the given language expression, which is a typical multi-modal task. Existing methods either adopt the pixel classification-based or the learnable query-based framework for mask generation, both of which are insufficient to deal with various text-image pairs with a fix number of parametric prototypes. In this work, we propose an end-to-end framework built on transformer to perform Linguistic query-Guided mask generation, dubbed LGFormer. It views the linguistic features as query to generate a specialized prot…
Learnable Shape Prototypes with Occlusion-Geometry-Guided Injection for Amodal Instance Segmentation
School of Mechanical Engineering and Automation, Northeastern University（机械工程与自动化学院，东北大学）；School of Information Science and Engineering, Northeastern University（信息科学与工程学院，东北大学）
Learning Multi-Modal Prototypes for Cross-Domain Few-Shot Object Detection
University of Chinese Academy of Sciences（中国科学院大学）；The Hong Kong Polytechnic University（香港理工大学）；Zhejiang University（浙江大学）；The University of Southern Queensland（昆士兰大学）
Cs2K: Class-specific and Class-shared Knowledge Guidance for Incremental Semantic Segmentation
Shenyang Institute of Automation, Chinese Academy of Sciences（中国科学院沈阳自动化研究所）；Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences（中国科学院机器人与智能制造研究院）；University of Chinese Academy of Sciences（中国科学院大学）；College of Automation Science and Engineering, South China University of Technology（华南理工大学自动化科学与工程学院）；Peking University（北京大学）
Center-guided Classifier for Semantic Segmentation of Remote Sensing Images
School of Software Technology, Zhejiang University（浙江大学软件学院）；Innovation Center of Yangtze River Delta, Zhejiang University（浙江大学长三角创新中心）；School of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；City University of Hong Kong（香港城市大学）
DTFormer: Text-Guided Semantic Alignment for RGB-D Segmentation
Southeast University（东南大学）；Technical University of Munich（慕尼黑工业大学）；City University of Macau（澳门城市大学）；University of Science and Technology of China（中国科学技术大学）
Prototype-Based Pseudo-Label Denoising for Source-Free Domain Adaptation in Remote Sensing Semantic Segmentation
College of Computer Science, Sichuan University（四川大学计算机科学学院）；College of Computer Science and Cyber Security, Chengdu University of Technology（成都理工大学计算机科学与网络安全学院）
We propose a fast, model agnostic method for finding interpretable counterfactual explanations of classifier predictions by using class prototypes. We show that class prototypes, obtained using either an encoder or through class specific k-d trees, significantly speed up the the search for counterfactual instances and result in more interpretable explanations. We introduce two novel metrics to quantitatively evaluate local interpretability at the instance level. We use these metrics to illustrate the effectiveness of our method on an image and tabular dataset, respectively MNIST and Breast Can…
SPOT-Occ: Sparse Prototype-guided Transformer for Camera-based 3D Occupancy Prediction
Artificial Intelligence Thrust, The Hong Kong University of Science and Technology (Guangzhou)（香港科学与技术大学人工智能方向（广州））；Department of Computer Science and Engineering, The Hong Kong University of Science and Technology（香港科学与技术大学计算机科学与工程系）
Nanjing University of Aeronautics & Astronautics（南京航空航天大学）
In the process of exploring the world, the curiosity constantly drives humans to cognize new things. Supposing you are a zoologist, for a presented animal image, you can recognize it immediately if you know its class. Otherwise, you would more likely attempt to cognize it by exploiting the side-information (e.g., semantic information, etc.) you have accumulated. Inspired by this, this paper decomposes the generalized zero-shot learning (G-ZSL) task into an open set recognition (OSR) task and a zero-shot learning (ZSL) task, where OSR recognizes seen classes (if we have seen (or known) them) an…
Seoul National University（首尔国立大学）；AIGENDRUG Co. Ltd.（AIGENDRUG有限公司）
While graph neural networks (GNNs) have been successful for node classification tasks and link prediction tasks in graph, learning graph-level representations still remains a challenge. For the graph-level representation, it is important to learn both representation of neighboring nodes, i.e., aggregation, and graph structural information. A number of graph pooling methods have been developed for this goal. However, most of the existing pooling methods utilize k-hop neighborhood without considering explicit structural information in a graph. In this paper, we propose Structure Prototype Guided…
Prototype-Guided Non-Exemplar Continual Learning for Cross-subject EEG Decoding
Dept. of Artificial Intelligence Korea University Seoul, Republic of Korea（人工智能系 韩国大学首尔共和国）；Cognitive Engineering Korea University Seoul, Republic of Korea（认知工程 韩国大学首尔共和国）
Continual learning (CL) refers to a machine learning paradigm that learns continuously without forgetting previously acquired knowledge. Thereby, major difficulty in CL is catastrophic forgetting of preceding tasks, caused by shifts in data distributions. Existing CL models often save a large number of old examples and stochastically revisit previously seen data to retain old knowledge. However, the occupied memory size keeps enlarging along with accumulating seen data. Hereby, we propose a memory-efficient CL method by storing a few samples to achieve good performance. We devise a dynamic pro…
Prototype-Guided and Lightweight Adapters for Inherent Interpretation and Generalisation in Federated Learning
Hertie Institute for AI in Brain Health, University of Tübingen, Germany（人工智能与脑健康赫尔蒂研究所，图宾根大学，德国）；Tübingen AI Center, University of Tübingen, Germany（图宾根人工智能中心，图宾根大学，德国）
Diagnosing Alzheimer's disease (AD) involves a deliberate diagnostic process owing to its innate traits of irreversibility with subtle and gradual progression. These characteristics make AD biomarker identification from structural brain imaging (e.g., structural MRI) scans quite challenging. Furthermore, there is a high possibility of getting entangled with normal aging. We propose a novel deep-learning approach through eXplainable AD Likelihood Map Estimation (XADLiME) for AD progression modeling over 3D sMRIs using clinically-guided prototype learning. Specifically, we establish a set of top…
QuantumPhaseNet: A Gauge-Covariant Geometric and Quantum-Spectral Theory of Semantic Concept Hierarchies with Prototype Validation of a Classical Quantum-Inspired Model
ProtoMol: Enhancing Molecular Property Prediction via Prototype-Guided Multimodal Learning
MBZUAI（穆扎芬人工智能研究所）；University of Zhengzhou（郑州大学）；HKUST（香港科技大学）；University of Aberdeen（爱丁堡大学）；MBZUAI, Weizmann Institute of Science（穆扎芬人工智能研究所、威斯曼科学研究所）
SCOPE: Structured Prototype-Guided Adaptation for EEG Foundation Models with Limited Labels
Saw Swee Hock School of Public Health, National University of Singapore（新加坡国立大学 Saw Swee Hock 公共卫生学院）；Institute of Data Science, National University of Singapore（新加坡国立大学数据科学研究所）；Guangzhou Research Translation and Innovation Institute, National University of Singapore（新加坡国立大学广州研究翻译与创新研究所）；College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算与数据科学学院）；Beijing Key Laboratory of Brainnetome and Brain-Computer Interface, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所脑网络与脑机接口重点实验室）；Brainnetome Center, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所脑网络中心）
Path-Guided Flow Matching for Dataset Distillation
Department of Machine Learning, MBZUAI, Abu Dhabi, UAE（机器学习系，MBZUAI，阿布扎赫德，阿联酋）；Department of Computer Vision, MBZUAI, Abu Dhabi, UAE（计算机视觉系，MBZUAI，阿布扎赫德，阿联酋）
DFAMS: Dynamic-flow guided Federated Alignment based Multi-prototype Search
National Engineering Research Center for Software Engineering, Peking University（软件工程国家工程研究中心，北京大学）；School of Computer Science, Peking University（北京大学计算机学院）；Key Laboratory of High Confidence Software Technologies, Ministry of Education（教育部高性能软件技术重点实验室）；Peking University Information Technology Institute (Tianjin Binhai)（北京大学信息科学技术研究院（天津滨海））
BrainSCL: Subtype-Guided Contrastive Learning for Brain Disorder Diagnosis
Computer Science and Engineering, Northeastern University, Shenyang, China（东北大学计算机科学与工程学院）；School of Computer Science and Artificial Intelligence, Shandong Normal University, Jinan, China（山东师范大学计算机科学与人工智能学院）；AiShiWeiLai AI Research, Beijing, China（艾世维来人工智能研究所）；Early Intervention Unit, Department of Psychiatry, Affiliated Nanjing Brain Hospital, Nanjing Medical University, Nanjing, China（南京医科大学附属南京脑科医院早期干预科）；Amii, University of Alberta, Edmonton, AB, Canada（阿尔伯塔大学Amii）
Explaining, Verifying, and Aligning Semantic Hierarchies in Vision-Language Model Embeddings
University of Lübeck（吕贝克大学）；Technical University of Munich（慕尼黑工业大学）；AUMOVIO SE；Osnabrück University（奥斯纳布吕克大学）；Leibniz Institute for Agricultural Engineering and Bioeconomy（莱布尼茨农业工程与生物经济研究所）；University of Hamburg（汉堡大学）
