Concept erasure is extensively utilized in image generation to prevent text-to-image models from generating undesired content. Existing methods can effectively erase narrow concepts that are specific and concrete, such as distinct intellectual properties (e.g. Pikachu) or recognizable characters (e.g. Elon Musk). However, their performance degrades on broad concepts such as ``sexual'' or ``violent'', whose wide scope and multi-faceted nature make them difficult to erase reliably. To overcome this limitation, we exploit the model's intrinsic embedding geometry to identify latent embeddings that encode a given concept. By clustering these embeddings, we derive a set of concept prototypes that summarize the model's internal representations of the concept, and employ them as negative conditioning signals during inference to achieve precise and reliable erasure. Extensive experiments across multiple benchmarks show that our approach achieves substantially more reliable removal of broad concepts while preserving overall image quality, marking a step towards safer and more controllable image generation.
Rethinking Robust Adversarial Concept Erasure in Diffusion Models
Beijing University of Posts and Telecommunications（北京邮电大学）；Dept. of Comp. Sci. and Tech., Institute for AI, Tsinghua University（计算机科学与技术系，人工智能研究院，清华大学）；University of Chinese Academy of Sciences（中国科学院大学）；Nanjing University of Aeronautics and Astronautics（南京航空航天大学）
Z-Erase: Enabling Concept Erasure in Single-Stream Diffusion Transformers
Beijing Advanced Innovation Center for Future Blockchain（未来区块链与隐私计算北京创新中心）；Privacy Computing, School of Artificial Intelligence, Beihang University（隐私计算，北京航空航天大学人工智能学院）；University of Science（科学大学）；Shanghai University（上海大学）；Tencent（腾讯）；Beijing Academy of Blockchain（北京区块链研究院）
SACE: Concept Erasure at the Semantic Singularity in Visual Autoregressive Models
School of Computer Science and Engineering, Sun Yat-sen University（中山大学计算机科学与工程学院）；School of Computer Science and Technology, Beijing Institute of Technology（北京理工大学计算机科学与技术学院）
TINA: Text-Free Inversion Attack for Unlearned Text-to-Image Diffusion Models
Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳））；City University of Hong Kong（香港城市大学）；Shenzhen Loop Area Institute（深圳河套学院）；Peng Cheng Laboratory（鹏城实验室）；Shandong University（山东大学）
Erased, But Not Forgotten: Erased Rectified Flow Transformers Still Remain Unsafe Under Concept Attack
Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing, School of Artificial Intelligence, Beihang University（北京航空航天大学人工智能学院未来区块链与隐私计算北京高精尖创新中心）；University of Science and Technology of China（中国科学技术大学）；National University of Defense Technology（国防科技大学）；GigaAI（极佳科技）；National University of Singapore（新加坡国立大学）
Semantic segmentation methods can not directly identify abnormal objects in images. Anomaly Segmentation algorithm from this realistic setting can distinguish between in-distribution objects and Out-Of-Distribution (OOD) objects and output the anomaly probability for pixels. In this paper, a Prototype Guided Anomaly segmentation Network (PGAN) is proposed to extract semantic prototypes for in-distribution training data from limited annotated images. In the model, prototypes are used to model the hierarchical category semantic information and distinguish OOD pixels. The proposed PGAN model incl…
SPENet: Self-guided Prototype Enhancement Network for Few-shot Medical Image Segmentation
School of Computer Science, Beijing University of Technology, Beijing, China（北京理工大学计算机科学学院）；Institute of Automation Chinese Academy of Sciences, Beijing, China（中国科学院自动化研究所）；Multimedia Department Xiaomi Inc, Beijing, China（小米公司多媒体部门）；Capital Medical University Affiliated Beijing Friendship Hospital, Beijing, China（首都医科大学附属北京友谊医院）
Diverse Normal Prototypes-Guided Contrastive Reconstruction for Medical Anomaly Detection
School of Mathematics, Shandong University（山东大学数学学院）；Department of Interventional Medicine and Minimally Invasive Oncology, The Second Qilu Hospital of Shandong University（山东大学第二齐鲁医院介入医学与微创肿瘤科）；Department of Neurosurgery, The Second Qilu Hospital of Shandong University（山东大学第二齐鲁医院神经外科）
Xi’an Jiaotong University（西安交通大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；University of California, Santa Cruz（加州大学圣克鲁兹分校）
The Diffusion Probabilistic Model (DPM) has emerged as a highly effective generative model in the field of computer vision. Its intermediate latent vectors offer rich semantic information, making it an attractive option for various downstream tasks such as segmentation and detection. In order to explore its potential further, we have taken a step forward and considered a more complex scenario in the medical image domain, specifically, under an unsupervised adaptation condition. To this end, we propose a Diffusion-based and Prototype-guided network (DP-Net) for unsupervised domain adaptive segm…
Peking University（北京大学）；South China University of Technology（华南理工大学）；National University of Singapore（新加坡国立大学）；MIT-IBM Watson AI Lab（麻省理工学院-IBM沃森人工智能实验室）
This paper studies a new, practical but challenging problem, called Class-Incremental Unsupervised Domain Adaptation (CI-UDA), where the labeled source domain contains all classes, but the classes in the unlabeled target domain increase sequentially. This problem is challenging due to two difficulties. First, source and target label sets are inconsistent at each time step, which makes it difficult to conduct accurate domain alignment. Second, previous target classes are unavailable in the current step, resulting in the forgetting of previous knowledge. To address this problem, we propose a nov…
Query-guided Prototype Evolution Network for Few-Shot Segmentation
Institute of Information Science, Beijing Jiaotong University（北京交通大学信息科学研究所）；School of Control Science and Engineering, Shandong University（山东大学控制科学与工程学院）；School of Computer Science and Technology, University of Chinese Academy of Sciences（中国科学院大学计算机科学与技术学院）；Department of Computer Science, City University of Hong Kong（香港城市大学计算机科学系）
Recent efforts towards video anomaly detection (VAD) try to learn a deep autoencoder to describe normal event patterns with small reconstruction errors. The video inputs with large reconstruction errors are regarded as anomalies at the test time. However, these methods sometimes reconstruct abnormal inputs well because of the powerful generalization ability of deep autoencoder. To address this problem, we present a novel approach for anomaly detection, which utilizes discriminative prototypes of normal data to reconstruct video frames. In this way, the model will favor the reconstruction of no…
Prototype-Guided Curriculum Learning for Zero-Shot Learning
School of Electronic Information and Communications, Huazhong University of Science and Technology（电子信息与通信学院，华中科技大学）；Mohamed bin Zayed University of AI（马尔代夫人工智能大学）；School of Computer Science and Engineering, Nanjing University of Science and Technology（计算机科学与工程学院，南京理工大学）；School of Computer Science, The University of Sydney（计算机科学学院，悉尼大学）
University of Edinburgh（爱丁堡大学）；Google Research（谷歌研究院）；Sungkyunkwan University（成均馆大学）
Prototype learning is extensively used for few-shot segmentation. Typically, a single prototype is obtained from the support feature by averaging the global object information. However, using one prototype to represent all the information may lead to ambiguities. In this paper, we propose two novel modules, named superpixel-guided clustering (SGC) and guided prototype allocation (GPA), for multiple prototype extraction and allocation. Specifically, SGC is a parameter-free and training-free approach, which extracts more representative prototypes by aggregating similar feature vectors, while GPA…
Institute of Computing Technology, CAS（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Beijing Academy of Artificial Intelligence（北京智源人工智能研究院）
Teaching machines to recognize a new category based on few training samples especially only one remains challenging owing to the incomprehensive understanding of the novel category caused by the lack of data. However, human can learn new classes quickly even given few samples since human can tell what discriminative features should be focused on about each category based on both the visual and semantic prior knowledge. To better utilize those prior knowledge, we propose the SEmantic Guided Attention (SEGA) mechanism where the semantic knowledge is used to guide the visual perception in a top-d…
Semantically Guided Dynamic Visual Prototype Refinement for Compositional Zero-Shot Learning
National Key Laboratory of Radar Signal Processing, Xidian University（雷达信号处理国家级重点实验室，西安电子科技大学）；Research Institute of Systems Engineering, Academy of Military Science（系统工程研究所，军事科学院）
Toward Realistic Remote Sensing Dataset Distillation with Discriminative Prototype-guided Diffusion
Computer Vision and Learning Systems, Department of Electrical Engineering, Linköping University（计算机视觉与学习系统，电气工程系，利厄普大学）；Helmholtz-Zentrum Dresden-Rossendorf, Responsible AI Group（海姆霍尔茨·德累斯顿-罗斯托克研究中心，负责任人工智能小组）；University of Iceland, Faculty of Electrical and Computer Engineering（冰岛大学，电气与计算机工程学院）
Recently, large-scale pre-trained models have shown their advantages in many tasks. However, due to the huge computational complexity and storage requirements, it is challenging to apply the large-scale model to real scenes. A common solution is knowledge distillation which regards the large-scale model as a teacher model and helps to train a small student model to obtain a competitive performance. Cross-task Knowledge distillation expands the application scenarios of the large-scale pre-trained model. Existing knowledge distillation works focus on directly mimicking the final prediction or th…
DualProtoSeg: Simple and Efficient Design with Text- and Image-Guided Prototype Learning for Weakly Supervised Histopathology Image Segmentation
University of Houston（德克萨斯大学休斯顿分校）；Ho Chi Minh City University of Technology（胡志明市技术大学）；University of Information Technology（信息技术大学）；University of Technology Sydney（悉尼技术大学）；Vin University（文大学）；Department of Computer Science, Emory University（埃默里大学计算机科学系）；College of Medical Science and Technology, Taipei Medical University（台北医学院医学科学与技术学院）；Montefiore Medical Center, Albert Einstein College of Medicine（蒙特福医院，阿尔伯特·爱因斯坦医学院）；School of Computer Science, Carnegie Mellon University（卡内基梅隆大学计算机科学学院）
PILoRA: Prototype Guided Incremental LoRA for Federated Class-Incremental Learning
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Centre for Artificial Intelligence and Robotics, Hong Kong Institute of Science and Innovation, Chinese Academy of Sciences（中国科学院香港创新研究院人工智能与机器人中心）
Data-Free Class-Incremental Gesture Recognition with Prototype-Guided Pseudo Feature Replay
School of Computer Science and Engineering, Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications, Ministry of Education, Southeast University（东南大学计算机科学与工程学院、教育部新一代人工智能技术及其跨学科应用重点实验室）；School of Cyber Science and Engineering, Engineering Research Center of Blockchain Application, Supervision And Management (Southeast University), Ministry of Education, Southeast University（东南大学网络空间安全学院、教育部区块链应用监管工程研究中心（东南大学））；New Laboratory of Pattern Recognition (NLPR), State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS), Institute of Automation, Chinese Academy of Sciences (CASIA)（中国科学院自动化研究所模式识别国家重点实验室、多模态人工智能系统全国重点实验室）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
Boosting Few-shot 3D Point Cloud Segmentation via Query-Guided Enhancement
Harbin Institute of Technology Shenzhen（哈尔滨工业大学深圳校区）；SmartMore（思谋科技）；Guangdong Provincial Key Laboratory of Novel Security Intelligence Technologies（广东省新型安全智能技术重点实验室）
Zero-Shot Skeleton-Based Action Recognition With Prototype-Guided Feature Alignment
School of Software Engineering, South China University of Technology（南方科技大学软件工程学院）；South China University of Technology（南方科技大学）；Pazhou Lab（琶洲实验室）；School of Future Technology, South China University of Technology（未来技术学院）；Peng Cheng Laboratory（鹏城实验室）；Key Laboratory of Big Data and Intelligent Robot (South China University of Technology), Ministry of Education（大数据与智能机器人重点实验室）
AI4Health Division, Faculty VI Carl von Ossietzky Universität Oldenburg（AI4Health部门，第六学院卡尔·冯·奥西特齐克大学）；Semantic Information Systems Group Osnabrück University（语义信息系统小组奥尔纳布里克大学）
ProBAG: Prototype-Guided Boundary-Aware Graph Diffusion for Weakly Supervised Histopathology Segmentation
AI VIETNAM Lab（AI越南实验室）；Washington University School of Medicine（华盛顿大学医学院）；Jeonbuk National University（全北国立大学）；Perelman School of Medicine, University of Pennsylvania（宾夕法尼亚大学佩雷尔曼医学院）
