Enhancing Object Detection with Privileged Information: A Model-Agnostic Teacher-Student Approach
Department of Artificial Intelligence, Faculty of Information and Communications Technology, University of Malta（人工智能系，信息与通讯技术学院，马耳他大学）；Department of Communications and Computer Engineering, Faculty of Information and Communications Technology, University of Malta（通讯与计算机工程系，信息与通讯技术学院，马耳他大学）；Department of Criminology, Faculty of Social Wellbeing, University of Malta（犯罪学系，社会福祉学院，马耳他大学）
Matthias Bartolo, Dylan Seychell, Gabriel Hili, Matthew Montebello, Carl James Debono, Saviour Formosa, Konstantinos Makantasis
This paper investigates the integration of the Learning Using Privileged Information (LUPI) paradigm in object detection to exploit fine-grained, descriptive information available during training but not at inference. We introduce a general, model-agnostic methodology for injecting privileged information-such as bounding box masks, saliency maps, and depth cues-into deep learning-based object detectors through a teacher-student architecture. Experiments are conducted across five state-of-the-art object detection models and multiple public benchmarks, including UAV-based litter detection datasets and Pascal VOC 2012, to assess the impact on accuracy, generalization, and computational efficiency. Our results demonstrate that LUPI-trained students consistently outperform their baseline counterparts, achieving significant boosts in detection accuracy with no increase in inference complexity or model size. Performance improvements are especially marked for medium and large objects, while ablation studies reveal that intermediate weighting of teacher guidance optimally balances learning from privileged and standard inputs. The findings affirm that the LUPI framework provides an effective and practical strategy for advancing object detection systems in both resource-constrained and real-world settings.
Conventional transfer learning leverages weights of pre-trained networks, but mandates the need for similar neural architectures. Alternatively, knowledge distillation can transfer knowledge between heterogeneous networks but often requires access to the original training data or additional generative networks. Knowledge transfer between networks can be improved by being agnostic to the choice of network architecture and reducing the dependence on original training data. We propose a knowledge transfer approach from a teacher to a student network wherein we train the student on an independent …
OmniOPSD: Rationale-Privileged On-Policy Self-Distillation for Affective Computing
Shenzhen University（深圳大学）；Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ)（广东省人工智能与数字经济实验室（深圳））；Tsinghua University（清华大学）；Shanghai Jiao Tong University（上海交通大学）；University of Science and Technology of China（中国科学技术大学）；Shenzhen Technology University（深圳技术大学）；Tongji University（同济大学）；Huawei（华为）
Online Action Detection (OAD) in videos is proposed as a per-frame labeling task to address the real-time prediction tasks that can only obtain the previous and current video frames. This paper presents a novel learning-with-privileged based framework for online action detection where the future frames only observable at the training stages are considered as a form of privileged information. Knowledge distillation is employed to transfer the privileged information from the offline teacher to the online student. We note that this setting is different from conventional KD because the difference …
Convolutional neural networks (CNNs) have allowed remarkable advances in single image super-resolution (SISR) over the last decade. Most SR methods based on CNNs have focused on achieving performance gains in terms of quality metrics, such as PSNR and SSIM, over classical approaches. They typically require a large amount of memory and computational units. FSRCNN, consisting of few numbers of convolutional layers, has shown promising results, while using an extremely small number of network parameters. We introduce in this paper a novel distillation framework, consisting of teacher and student …
Privileged Lesion-Context Relational Distillation for Mask-Free Skin Lesion Classification
Victoria University（维多利亚大学）；Bangaldesh University of Business and Technology（孟加拉国商业与技术大学）；Daffodil Institute of Information Technology（达芙妮信息技术学院）；American International University-Bangladesh (AIUB)（孟加拉国美国国际大学）；Multimedia University（多媒体大学）
University of Maryland, College Park（马里兰大学帕克分校）；Georgia Institute of Technology（佐治亚理工学院）；Johns Hopkins University（约翰斯·霍普金斯大学）；University of Oxford（牛津大学）；UC San Diego（加利福尼亚大学圣迭戈分校）
Beijing University of Posts and Telecommunications（北京邮电大学）；OPPO Research Institute（OPPO研究院）；Xmotors；University of Maryland, Baltimore County（马里兰大学巴尔的摩县分校）
Performance of trimap-free image matting methods is limited when trying to decouple the deterministic and undetermined regions, especially in the scenes where foregrounds are semantically ambiguous, chromaless, or high transmittance. In this paper, we propose a novel framework named Privileged Prior Information Distillation for Image Matting (PPID-IM) that can effectively transfer privileged prior environment-aware information to improve the performance of students in solving hard foregrounds. The prior information of trimap regulates only the teacher model during the training stage, while not…
Simon Fraser University（西蒙弗雷泽大学）；Borealis AI（Borealis人工智能研究院）
Knowledge distillation is a widely applicable technique for training a student neural network under the guidance of a trained teacher network. For example, in neural network compression, a high-capacity teacher is distilled to train a compact student; in privileged learning, a teacher trained with privileged data is distilled to train a student without access to that data. The distillation loss determines how a teacher's knowledge is captured and transferred to the student. In this paper, we propose a new form of knowledge distillation loss that is inspired by the observation that semantically…
OPD-V: Visual On-Policy Self-Distillation with Modality Balance
National University of Singapore（新加坡国立大学）；Ludwig Maximilian University of Munich（慕尼黑大学）；Munich Center for Machine Learning（慕尼黑机器学习中心）；Sun Yat-sen University（中山大学）
Siemens AG, Corporate Technology（西门子公司企业技术部）；University of Munich（慕尼黑大学）
In recent years, many explanation methods have been proposed to explain individual classifications of deep neural networks. However, how to leverage the created explanations to improve the learning process has been less explored. As the privileged information, the explanations of a model can be used to guide the learning process of the model itself. In the community, another intensively investigated privileged information used to guide the training of a model is the knowledge from a powerful teacher model. The goal of this work is to leverage the self-explanation to improve the learning proces…
JDCNet: Confidence-Gated Privileged-Modality Distillation for Cost-Preserving X-ray Inference
Auckland University of Technology（奥克兰技术大学）；Guilin University of Electronic Technology（桂林电子科技大学）；Hikvision Technology Co., Ltd（海康威视技术有限公司）；Hebei University of Technology（河北工业大学）
VAD: Attributing Visual Evidence for Target Reconstruction in Multimodal On-Policy Distillation
Shanghai Jiao Tong University（上海交通大学）；Xiaohongshu Inc.（小红书公司）；The Chinese University of Hong Kong（香港中文大学）；Zhejiang University（浙江大学）；Southeast University（东南大学）
Hefei University of Technology（合肥工业大学）；UBTech Sydney Artificial Intelligence Institute（优必选悉尼人工智能研究院）；The University of Sydney（悉尼大学）
Despite the promising progress made in recent years, person re-identification remains a challenging task due to complex variations in human appearances from different camera views. This paper presents a logistic discriminant metric learning method for this challenging problem. Different with most existing metric learning algorithms, it exploits both original data and auxiliary data during training, which is motivated by the new machine learning paradigm - Learning Using Privileged Information. Such privileged information is a kind of auxiliary knowledge which is only available during training.…
The primary goal of knowledge distillation (KD) is to encapsulate the information of a model learned from a teacher network into a student network, with the latter being more compact than the former. Existing work, e.g., using Kullback-Leibler divergence for distillation, may fail to capture important structural knowledge in the teacher network and often lacks the ability for feature generalization, particularly in situations when teacher and student are built to address different classification tasks. We propose Wasserstein Contrastive Representation Distillation (WCoRD), which leverages both…
From Draft to Draft-Free: One-Step Video Object Removal via Privileged Distillation and Fast Planting
Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University（西安交通大学人工智能与机器人研究所）；SGIT AI Lab, State Grid Corporation of China（国家电网公司SGIT人工智能实验室）；Zhejiang University of Technology（浙江工业大学）；Baidu（百度）
Visual-OPSD: Cross-Modal On-Policy Self-Distillation for Efficient Unified Multimodal Reasoning
Xi’an Jiaotong University（西安交通大学）；MOE KLINNS Lab（MOE KLINNS实验室）；Shaanxi Province Key Laboratory of Big Data Knowledge Engineering（陕西省大数据知识工程重点实验室）；Sun Yat-sen University（中山大学）
RS-OPSD: Reliable Privileged On-Policy-Self-Distillation for Ultra-High-Resolution Remote Sensing VQA
Tsinghua University（清华大学）；Zhejiang University（浙江大学）；Central University of Finance and Economics（中央财经大学）；East China Normal University（华东师范大学）；Key Laboratory of Geographic Information Science（地理信息科学重点实验室）
