Curating datasets for object segmentation is a difficult task. With the advent of large-scale pre-trained generative models, conditional image generation has been given a significant boost in result quality and ease of use. In this paper, we present a novel method that enables the generation of general foreground-background segmentation models from simple textual descriptions, without requiring segmentation labels. We leverage and explore pre-trained latent diffusion models, to automatically generate weak segmentation masks for concepts and objects. The masks are then used to fine-tune the diffusion model on an inpainting task, which enables fine-grained removal of the object, while at the same time providing a synthetic foreground and background dataset. We demonstrate that using this method beats previous methods in both discriminative and generative performance and closes the gap with fully supervised training while requiring no pixel-wise object labels. We show results on the task of segmenting four different objects (humans, dogs, cars, birds) and a use case scenario in medical image analysis. The code is available at https://github.com/MischaD/fobadiffusion.
This paper proposes a knowledge distillation method for foreground object search (FoS). Given a background and a rectangle specifying the foreground location and scale, FoS retrieves compatible foregrounds in a certain category for later image composition. Foregrounds within the same category can be grouped into a small number of patterns. Instances within each pattern are compatible with any query input interchangeably. These instances are referred to as interchangeable foregrounds. We first present a pipeline to build pattern-level FoS dataset containing labels of interchangeable foregrounds…
FSD-BEV: Foreground Self-Distillation for Multi-view 3D Object Detection
Hangzhou Innovation Institute, Beihang University（北京航空航天大学杭州创新研究院）；School of Software, Beihang University（北京航空航天大学软件学院）；State Key Laboratory of Virtual Reality Technology and Systems, Beihang University（北京航空航天大学虚拟现实技术与系统国家重点实验室）；Shanghai ZEEKR Blue New Energy Technology Co., Ltd.（上海极氪蓝新能源科技有限公司）；Zhongguancun Laboratory, Beijing, China（中关村实验室）
Foreground-Aware Dataset Distillation via Dynamic Patch Selection
Graduate School of Information Science and Technology, Hokkaido University（信息科学与技术研究生学校，北海道大学）；Education and Research Center for Mathematical and Data Science, Hokkaido University（数学与数据科学教育与研究中心，北海道大学）；Faculty of Information Science and Technology, Hokkaido University（信息科学与技术学部，北海道大学）
Generalizable Knowledge Distillation from Vision Foundation Models for Semantic Segmentation
School of Artificial Intelligence, Xidian University, China（西安电子科技大学人工智能学院）；Department of Information Engineering and Computer Science, University of Trento, Italy（特伦托大学信息工程与计算机科学系）；Department of Automation, Tsinghua University, China（清华大学自动化系）；School of Computer Science and Information Engineering, Hefei University of Technology, China（合肥工业大学计算机科学与信息工程学院）
Pathryoshka: Compressing Pathology Foundation Models via Multi-Teacher Knowledge Distillation with Nested Embeddings
Technical University of Munich（慕尼黑技术大学）；Helmholtz Munich（马克斯·普朗克研究所慕尼黑分部）；Munich Data Science Institute（慕尼黑数据科学研究所）；Munich Center for Machine Learning（慕尼黑机器学习中心）
Beijing University of Posts and Telecommunications（北京邮电大学）；OPPO Research Institute（OPPO研究院）；Xmotors；University of Maryland, Baltimore County（马里兰大学巴尔的摩县分校）
Performance of trimap-free image matting methods is limited when trying to decouple the deterministic and undetermined regions, especially in the scenes where foregrounds are semantically ambiguous, chromaless, or high transmittance. In this paper, we propose a novel framework named Privileged Prior Information Distillation for Image Matting (PPID-IM) that can effectively transfer privileged prior environment-aware information to improve the performance of students in solving hard foregrounds. The prior information of trimap regulates only the teacher model during the training stage, while not…
Harnessing Adversarial Distillation to Customise Debiased, Disease-Specific Pathology Foundation Models for Breast Cancer
School of Computer Science and Engineering, South China University of Technology（华南理工大学计算机科学与工程学院）；School of Computing and Mathematical Sciences, University of Leicester（莱斯特大学计算与数学科学学院）；Leicester Cancer Research Centre, University of Leicester（莱斯特大学莱斯特癌症研究中心）；Department of Engineering Science, University of Oxford（牛津大学工程科学系）；Nuffield Department of Medicine, University of Oxford（牛津大学纳菲尔德医学院）；Department of Computer Science and Engineering, The Hong Kong University of Science and Technology（香港科技大学计算机科学与工程学系）；ZoyMed（佐医医疗（ZoyMed））
In the field of 3D object detection for autonomous driving, the sensor portfolio including multi-modality and single-modality is diverse and complex. Since the multi-modal methods have system complexity while the accuracy of single-modal ones is relatively low, how to make a tradeoff between them is difficult. In this work, we propose a universal cross-modality knowledge distillation framework (UniDistill) to improve the performance of single-modality detectors. Specifically, during training, UniDistill projects the features of both the teacher and the student detector into Bird's-Eye-View (BE…
Task-Specific Knowledge Distillation from the Vision Foundation Model for Enhanced Medical Image Segmentation
Shanghai Jiao Tong University School of Medicine（上海交通大学医学院）；Shanghai University（上海大学）；Sun Yat-sen University（中山大学）；The Hong Kong University of Science and Technology（香港科技大学）；East China Normal University（华东师范大学）；Shanghai Jiao Tong University（上海交通大学）
Bridging Cross-task Protocol Inconsistency for Distillation in Dense Object Detection
College of Computer Science & Technology, Zhejiang University（浙江大学计算机科学与技术学院）；Polytechnic Institute, Zhejiang University（浙江大学理工学院）；Hikvision Research Institute（海康威视研究院）；ZJU – UIUC Institute, Zhejiang University（浙江大学-伊利诺伊大学厄巴纳-香槟分校研究院）；Shanghai Institute for Advanced Study of Zhejiang University（浙江大学上海高等研究院）
Peking University（北京大学）；NIO（蔚来）；The Chinese University of Hong Kong（香港中文大学）；University of Chinese Academy of Sciences（中国科学院大学）
To achieve accurate and low-cost 3D object detection, existing methods propose to benefit camera-based multi-view detectors with spatial cues provided by the LiDAR modality, e.g., dense depth supervision and bird-eye-view (BEV) feature distillation. However, they directly conduct point-to-point mimicking from LiDAR to camera, which neglects the inner-geometry of foreground targets and suffers from the modal gap between 2D-3D features. In this paper, we propose the learning scheme of Target Inner-Geometry from the LiDAR modality into camera-based BEV detectors for both dense depth and BEV featu…
Unifying Multiple Foundation Models for Advanced Computational Pathology
Shanghai Jiao Tong University（上海交通大学）；Washington University in St. Louis（华盛顿大学）；University of Science and Technology Beijing（北京科技大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Department of Surgical Oncology and General Surgery, Key Laboratory of Precision Diagnosis and Treatment of Gastrointestinal Tumours, Ministry of Education, The First Hospital of China Medical University（外科肿瘤科和普通外科，国家教育委员会胃肠道肿瘤精准诊断与治疗重点实验室，中国医科大学第一医院）；Shanghai Innovation Institute（上海创新研究院）；Sensetime Research（商汤科技研究院）
Feasibility of Indoor Frame-Wise Lidar Semantic Segmentation via Distillation from Visual Foundation Model
1 Department of Earth Observation Science, Faculty of Geo-Information Science；Earth Observation (ITC), University of Twente, 7522 NB Enschede, The Netherlands
Multi-Teacher Contrastive Distillation for Edge-Efficient Pathology Foundation Models
EKFZ for Digital Health（数字健康EKFZ）；TU Dresden（德累斯顿工业大学）；University of Augsburg（奥格斯堡大学）；Bavarian Cancer Research Center (BZKF)（巴伐利亚癌症研究中心）；Department of Medicine I, TU Dresden（德累斯顿工业大学第一医学院）；NCT Heidelberg（海德堡NCT）；University of Leeds（利兹大学）
ConStruct: Structural Distillation of Foundation Models for Prototype-Based Weakly Supervised Histopathology Segmentation
Ho Chi Minh City University of Technology（胡志明市技术大学）；University of Technology Sydney（悉尼技术大学）；University of Houston（休斯顿大学）；University of Information Technology（信息科技大学）；College of Medical Science and Technology, Taipei Medical University（台北医学院医学科技学院）；Department of Computer Science, Emory University（埃默里大学计算机科学系）；Montefiore Medical Center, Albert Einstein College of Medicine（蒙特福伊医疗中心，阿尔伯特·爱因斯坦医学院）；School of Computer Science, Carnegie Mellon University（卡内基梅隆大学计算机科学学院）
Sharpening Lightweight Models for Generalized Polyp Segmentation: A Boundary Guided Distillation from Foundation Models
Dept. of Computer Science and Engineering（计算机科学与工程系）；Malaviya National Institute of Technology（马拉维亚国立理工学院）；INRIA Sophia Antipolis（INRIA索菲亚-安蒂波利斯研究中心）；Côte d'Azur University（蔚蓝海岸大学）
