Contextual information in human environments, such as signs, symbols, and objects provide important information for robots to use for exploration and navigation. To identify and segment contextual information from complex images obtained in these environments, data-driven methods such as Convolutional Neural Networks (CNNs) are used. However, these methods require large amounts of human labeled data which are slow and time-consuming to obtain. Weakly supervised methods address this limitation by generating pseudo segmentation labels (PSLs). In this paper, we present the novel Weakly Supervised Mask Data Distillation (WeSuperMaDD) architecture for autonomously generating PSLs using CNNs not specifically trained for the task of context segmentation; i.e., CNNs trained for object classification, image captioning, etc. WeSuperMaDD uniquely generates PSLs using learned image features from sparse and limited diversity data; common in robot navigation tasks in human-centred environments (malls, grocery stores). Our proposed architecture uses a new mask refinement system which automatically searches for the PSL with the fewest foreground pixels that satisfies cost constraints. This removes the need for handcrafted heuristic rules. Extensive experiments successfully validated the performance of WeSuperMaDD in generating PSLs for datasets with text of various scales, fonts, and perspectives in multiple indoor/outdoor environments. A comparison with Naive, GrabCut, and Pyramid methods found a significant improvement in label and segmentation quality. Moreover, a context segmentation CNN trained using the WeSuperMaDD architecture achieved measurable improvements in accuracy compared to one trained with Naive PSLs. Our method also had comparable performance to existing state-of-the-art text detection and segmentation methods on real datasets without requiring segmentation labels for training.
University of Electronic Science and Technology of China（电子科技大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；West China Hospital, Sichuan University（四川大学华西医院）
The success of Convolutional Neural Networks (CNNs) in 3D medical image segmentation relies on massive fully annotated 3D volumes for training that are time-consuming and labor-intensive to acquire. In this paper, we propose to annotate a segmentation target with only seven points in 3D medical images, and design a two-stage weakly supervised learning framework PA-Seg. In the first stage, we employ geodesic distance transform to expand the seed points to provide more supervision signal. To further deal with unannotated image regions during training, we propose two contextual regularization str…
Infrared UAV Target Tracking with Dynamic Feature Refinement and Global Contextual Attention Knowledge Distillation
School of Computer Science and Technology, Xidian University（西安电子科技大学计算机科学与技术学院）；Xi’an Modern Control Technology Research Institute（西安现代控制技术研究所）；School of Artificial Intelligence and Automation, Huazhong University of Science and Technology（华中科技大学人工智能与自动化学院）
LadderMIL: Multiple Instance Learning with Coarse-to-Fine Self-Distillation
Institute for Regeneration and Repair, University of Edinburgh（再生与修复研究所，爱丁堡大学）；School of Informatics, University of Edinburgh（信息学院，爱丁堡大学）；Indica Labs（Indica实验室）；School of Medicine, University of St Andrews（医学学院，圣安德鲁斯大学）；Department of Basic Pathology, National Defense Medical College（基础病理学部门，国家防卫医疗学院）
The Chinese University of Hong Kong（香港中文大学）；SenseTime Group Limited（商汤集团有限公司）；Nanyang Technological University（南洋理工大学）
Training deep models for lane detection is challenging due to the very subtle and sparse supervisory signals inherent in lane annotations. Without learning from much richer context, these models often fail in challenging scenarios, e.g., severe occlusion, ambiguous lanes, and poor lighting conditions. In this paper, we present a novel knowledge distillation approach, i.e., Self Attention Distillation (SAD), which allows a model to learn from itself and gains substantial improvement without any additional supervision or labels. Specifically, we observe that attention maps extracted from a model…
In video understanding, most cross-modal knowledge distillation (KD) methods are tailored for classification tasks, focusing on the discriminative representation of the trimmed videos. However, action detection requires not only categorizing actions, but also localizing them in untrimmed videos. Therefore, transferring knowledge pertaining to temporal relations is critical for this task which is missing in the previous cross-modal KD frameworks. To this end, we aim at learning an augmented RGB representation for action detection, taking advantage of additional modalities at training time throu…
Most existing salient object detection (SOD) models are difficult to apply due to the complex and huge model structures. Although some lightweight models are proposed, the accuracy is barely satisfactory. In this paper, we design a novel semantics-guided contextual fusion network (SCFNet) that focuses on the interactive fusion of multi-level features for accurate and efficient salient object detection. Furthermore, we apply knowledge distillation to SOD task and provide a sizeable dataset KD-SOD80K. In detail, we transfer the rich knowledge from a seasoned teacher to the untrained SCFNet throu…
Knowledge Distillation (KD) is a strategy for the definition of a set of transferability gangways to improve the efficiency of Convolutional Neural Networks. Feature-based Knowledge Distillation is a subfield of KD that relies on intermediate network representations, either unaltered or depth-reduced via maximum activation maps, as the source knowledge. In this paper, we propose and analyse the use of a 2D frequency transform of the activation maps before transferring them. We pose that\textemdash by using global image cues rather than pixel estimates, this strategy enhances knowledge transfer…
University of Pisa（比萨大学）；Scuola Normale Superiore（比萨高等师范学校）；Computer Vision Center（计算机视觉中心）
Distributed learning on the edge often comprises self-centered devices (SCD) which learn local tasks independently and are unwilling to contribute to the performance of other SDCs. How do we achieve forward transfer at zero cost for the single SCDs? We formalize this problem as a Distributed Continual Learning scenario, where SCD adapt to local tasks and a CL model consolidates the knowledge from the resulting stream of models without looking at the SCD's private data. Unfortunately, current CL methods are not directly applicable to this scenario. We propose Data-Agnostic Consolidation (DAC), …
Privileged Lesion-Context Relational Distillation for Mask-Free Skin Lesion Classification
Victoria University（维多利亚大学）；Bangaldesh University of Business and Technology（孟加拉国商业与技术大学）；Daffodil Institute of Information Technology（达芙妮信息技术学院）；American International University-Bangladesh (AIUB)（孟加拉国美国国际大学）；Multimedia University（多媒体大学）
One-for-All: Bridge the Gap Between Heterogeneous Architectures in Knowledge Distillation
School of information and Electronics, Beijing Institute of Technology（北京理工大学信息与电子学院）；Huawei Noah’s Ark Lab（华为诺亚方舟实验室）；School of Computer Science, Faculty of Engineering, The University of Sydney（悉尼大学工程学院计算机科学学院）
ReCo-KD: Region- and Context-Aware Knowledge Distillation for Efficient 3D Medical Image Segmentation
McWilliams School of Biomedical Informatics, The University of Texas Health Science Center at Houston (UTHealth Houston)（麦威廉生物医学信息学学院，德克萨斯大学健康科学中心休斯顿分校（UTHealth休斯顿））
School of Computer Science, Wuhan University（武汉大学计算机学院）；School of Computer Science, The University of Sydney（悉尼大学计算机学院）；School of Computer Science and Engineering, Nanyang Technological University（南洋理工大学计算机科学与工程学院）
Unleashing the Potential of Mamba: Boosting a LiDAR 3D Sparse Detector by Using Cross-Model Knowledge Distillation
East China University of Science and Technology（华东理工大学）；University of Sydney（悉尼大学）；College of Computer Science and Artificial Intelligence, Fudan University（复旦大学计算机科学技术学院）
COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-training
Technical University of Munich（慕尼黑工业大学）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；Helmholtz Munich（亥姆霍兹慕尼黑中心）；Munich Data Science Institute (MDSI)（慕尼黑数据科学研究所）
Holistic White-light Polyp Classification via Alignment-free Dense Distillation of Auxiliary Optical Chromoendoscopy
Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology（光电国立实验室、华中科技大学）；Changzhou United Imaging Surgical Co., Ltd（常州联合影像手术有限公司）；Tongji Medical College of Huazhong University of Science and Technology（华中科技大学同济医学院）
Graduate School of Advanced Science and Engineering Hiroshima University（广岛大学先进科学与工程学院）；Intelligent Production Technology Research & Development Center for Aerospace Gifu University（岐阜大学航空航天智能生产技术研究开发中心）
Online social networks contain a constantly increasing amount of images - most of them focusing on people. Due to cultural and climate factors, fashion trends and physical appearance of individuals differ from city to city. In this paper we investigate to what extent such cues can be exploited in order to infer the geographic location, i.e. the city, where a picture was taken. We conduct a user study, as well as an evaluation of automatic methods based on convolutional neural networks. Experiments on the Fashion 144k and a Pinterest-based dataset show that the automatic methods succeed at this…
A Coarse-to-Fine Human Pose Estimation Method based on Two-stage Distillation and Progressive Graph Neural Network
Key Laboratory of Computational Intelligence and Chinese Information Processing of Ministry of Education, School of Computer and Information Technology, Shanxi University（教育部计算智能与中文信息处理重点实验室，计算机与信息技术学院，山西大学）
Zhejiang University-University of Illinois at Urbana-Champaign Institute, Zhejiang University（浙江大学伊利诺伊大学厄巴纳香槟校区联合学院，浙江大学）
Recently, much progress has been made for self-supervised action recognition. Most existing approaches emphasize the contrastive relations among videos, including appearance and motion consistency. However, two main issues remain for existing pre-training methods: 1) the learned representation is neutral and not informative for a specific task; 2) multi-task learning-based pre-training sometimes leads to sub-optimal solutions due to inconsistent domains of different tasks. To address the above issues, we propose a novel action recognition pre-training framework, which exploits human-centered p…
Gated Relational Alignment via Confidence-based Distillation for Efficient VLMs
Department of Information Technology（信息科技系）；Electrical Engineering, ETH Zurich, Zurich, Switzerland（电气工程，苏黎世联邦理工学院，苏黎世，瑞士）；Qualcomm AI Research, Amsterdam, the Netherlands（高通人工智能研究，阿姆斯特丹，荷兰）；Department of Electrical, Electronic and Information Engineering（电气、电子与信息工程系）；University of Bologna, Bologna, Italy（博洛尼亚大学，博洛尼亚，意大利）；School of Electrical and Electronic Engineering（电气与电子工程学院）
Current referring expression comprehension algorithms can effectively detect or segment objects indicated by nouns, but how to understand verb reference is still under-explored. As such, we study the challenging problem of task oriented detection, which aims to find objects that best afford an action indicated by verbs like sit comfortably on. Towards a finer localization that better serves downstream applications like robot interaction, we extend the problem into task oriented instance segmentation. A unique requirement of this task is to select preferred candidates among possible alternative…
