虽然基于深度学习的图像恢复已实现前所未有的保真度，但在移动神经处理单元（NPU）上的部署仍受限于算子不兼容和内存访问开销。我们提出了一种NPU感知的硬件-算法协同设计方法，用于移动NPU上的真实图像去噪。该方法采用高容量教师网络监督轻量级学生网络，该学生网络专门设计以利用现代移动SoC的平铺内存架构。通过优先使用NPU原生原语——标准3x3卷积、ReLU激活和最近邻上采样——并采用渐进式上下文扩展策略（高达1024x1024裁剪），该模型在Mobile AI 2026挑战赛的全分辨率（2432x3200）验证基准上达到37.66 dB PSNR / 0.9278 SSIM，在保留测试基准上达到37.58 dB PSNR / 0.9098 SSIM。根据官方挑战规则，在标准全高清（1088x1920）协议下测量推理时间，在联发科天玑9500上为34.0毫秒，在高通骁龙8 Elite NPU上为46.1毫秒。我们进一步揭示了“推理反转”效应，即严格遵循NPU兼容操作可使专用NPU执行速度比集成移动GPU快3.88倍。1.96M参数的学生网络通过高alpha知识蒸馏（alpha=0.9）恢复了教师网络99.8%的恢复质量，实现了21.2倍的参数缩减，同时将PSNR差距从1.63 dB缩小至仅0.05 dB。这些结果确立了硬件感知蒸馏作为统一高保真去噪与跨多种移动NPU架构实际部署的有效策略。所提出的轻量级学生网络（LiteDenoiseNet）及其训练统计数据已在NN数据集中提供，网址为https://github.com/ABrain-One/NN-Dataset。
英文摘要：
While deep-learning-based image restoration has achieved unprecedented fidelity, deployment on mobile Neural Processing Units (NPUs) remains bottlenecked by operator incompatibility and memory-access overhead. We propose an NPU-aware hardware-algorithm co-design approach for real-world image denoising on mobile NPUs. Our approach employs a high-capacity teacher to supervise a lightweight student network specifically designed to leverage the tiled-memory architectures of modern mobile SoCs. By prioritizing NPU-native primitives -- standard 3x3 convolutions, ReLU activations, and nearest-neighbor upsampling -- and employing a progressive context expansion strategy (up to 1024x1024 crops), the model achieves 37.66 dB PSNR / 0.9278 SSIM on the validation benchmark and 37.58 dB PSNR / 0.9098 SSIM on the held-out test benchmark at full resolution (2432x3200) in the Mobile AI 2026 challenge. Following the official challenge rules, the inference runtime is measured under a standardized Full HD (1088x1920) protocol, where it runs in 34.0 ms on the MediaTek Dimensity 9500 and 46.1 ms on the Qualcomm Snapdragon 8 Elite NPU. We further reveal an "Inference Inversion" effect, where strict adherence to NPU-compatible operations enables dedicated NPU execution up to 3.88x faster than the integrated mobile GPU. The 1.96M-parameter student recovers 99.8% of the teacher's restoration quality via high-alpha knowledge distillation (alpha = 0.9), achieving a 21.2x parameter reduction while closing the PSNR gap from 1.63 dB to only 0.05 dB. These results establish hardware-aware distillation as an effective strategy for unifying high-fidelity denoising with practical deployment across diverse mobile NPU architectures. The proposed lightweight student model (LiteDenoiseNet) and its training statistics are provided in the NN Dataset, available at https://github.com/ABrain-One/NN-Dataset.
Iterative generative models, such as noise conditional score networks and denoising diffusion probabilistic models, produce high quality samples by gradually denoising an initial noise vector. However, their denoising process has many steps, making them 2-3 orders of magnitude slower than other generative models such as GANs and VAEs. In this paper, we establish a novel connection between knowledge distillation and image generation with a technique that distills a multi-step denoising process into a single step, resulting in a sampling speed similar to other single-step generative models. Our …
We propose an efficient neural network for RAW image denoising. Although neural network-based denoising has been extensively studied for image restoration, little attention has been given to efficient denoising for compute limited and power sensitive devices, such as smartphones and smartwatches. In this paper, we present a novel architecture and a suite of training techniques for high quality denoising in mobile devices. Our work is distinguished by three main contributions. (1) Feature-Align layer that modulates the activations of an encoder-decoder architecture with the input noisy images. …
The Chinese University of Hong Kong（香港中文大学）；East China Normal University（华东师范大学）；Nanjing University of Posts and Telecommunications（南京邮电大学）
Single image denoising (SID) has achieved significant breakthroughs with the development of deep learning. However, the proposed methods are often accompanied by plenty of parameters, which greatly limits their application scenarios. Different from previous works that blindly increase the depth of the network, we explore the degradation mechanism of the noisy image and propose a lightweight Multiple Degradation and Reconstruction Network (MDRN) to progressively remove noise. Meanwhile, we propose two novel Heterogeneous Knowledge Distillation Strategies (HMDS) to enable MDRN to learn richer an…
Steklov Mathematical Institute（斯捷克洛夫数学研究所）；Neuromation OU；Samsung AI Center（三星人工智能中心）
Despite recent advances in computer vision based on various convolutional architectures, video understanding remains an important challenge. In this work, we present and discuss a top solution for the large-scale video classification (labeling) problem introduced as a Kaggle competition based on the YouTube-8M dataset. We show and compare different approaches to preprocessing, data augmentation, model architectures, and model combination. Our final model is based on a large ensemble of video- and frame-level models but fits into rather limiting hardware constraints. We apply an approach based …
Harbin Institute of Technology（哈尔滨工业大学）；University of Tianjin（天津大学）；Peng Cheng Lab（鹏城实验室）
We investigate the task of learning blind image denoising networks from an unpaired set of clean and noisy images. Such problem setting generally is practical and valuable considering that it is feasible to collect unpaired noisy and clean images in most real-world applications. And we further assume that the noise can be signal dependent but is spatially uncorrelated. In order to facilitate unpaired learning of denoising network, this paper presents a two-stage scheme by incorporating self-supervised learning and knowledge distillation. For self-supervised learning, we suggest a dilated blind…
Denoising Diffusion models have demonstrated their proficiency for generative sampling. However, generating good samples often requires many iterations. Consequently, techniques such as binary time-distillation (BTD) have been proposed to reduce the number of network calls for a fixed architecture. In this paper, we introduce TRAnsitive Closure Time-distillation (TRACT), a new method that extends BTD. For single step diffusion,TRACT improves FID by up to 2.4x on the same architecture, and achieves new single-step Denoising Diffusion Implicit Models (DDIM) state-of-the-art FID (7.4 for ImageNet…
For tabular data sets, we explore data and model distillation, as well as data denoising. These techniques improve both gradient-boosting models and a specialized DNN architecture. While gradient boosting is known to outperform DNNs on tabular data, we close the gap for datasets with 100K+ rows and give DNNs an advantage on small data sets. We extend these results with input-data distillation and optimized ensembling to help DNN performance match or exceed that of gradient boosting. As a theoretical justification of our practical method, we prove its equivalence to classical cross-entropy know…
Adversarial Robustness Distillation (ARD) is a novel method to boost the robustness of small models. Unlike general adversarial training, its robust knowledge transfer can be less easily restricted by the model capacity. However, the teacher model that provides the robustness of knowledge does not always make correct predictions, interfering with the student's robust performances. Besides, in the previous ARD methods, the robustness comes entirely from one-to-one imitation, ignoring the relationship between examples. To this end, we propose a novel structured ARD method called Contrastive Rela…
The goal of this work is to investigate what singing voice separation approaches based on neural networks learn from the data. We examine the mapping functions of neural networks based on the denoising autoencoder (DAE) model that are conditioned on the mixture magnitude spectra. To approximate the mapping functions, we propose an algorithm inspired by the knowledge distillation, denoted the neural couplings algorithm (NCA). The NCA yields a matrix that expresses the mapping of the mixture to the target source magnitude information. Using the NCA, we examine the mapping functions of three fund…
PFCM: Poisson flow consistency models for low-dose CT image denoising
Stanford University（斯坦福大学）；KTH Royal Institute of Technology（瑞典皇家理工学院）；Karolinska University Hospital（卡罗林斯卡大学医院）；GE HealthCare（GE医疗）；Rensselaer Polytechnic Institute（伦斯勒理工学院）
Denoising Mutual Knowledge Distillation in Bi-Directional Multiple Instance Learning
College of Computer Science, Sichuan University（四川大学计算机学院）；Sichuan University-Pittsburgh Institute（四川大学匹兹堡研究院）；Department of Pathology West China Hospital, Sichuan University（四川大学华西医院病理科部门）；Institute of Clinical Pathology, West China Hospital, Sichuan University（四川大学华西医院临床病理研究所）
Bridging Diffusion Pruning and Step Distillation with Teacher-Aligned Repair
School of Big Data and Artificial Intelligence（大数据与人工智能学院）；Guangdong University of Finance and Economics（广东财经大学）；School of Computer Science（计算机科学学院）；Shandong University（山东大学）；GDUFE-UCM Joint Institute（广东财经大学-UCM联合学院）
FocusOPD: Structure-Guided On-Policy Distillation for Diffusion Models
University of Science and Technology of China（中国科学技术大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Huawei Noah’s Ark Lab（华为诺亚方舟实验室）
When Distillation Breaks Motion Control: Restoring Generative Trajectories for Fast Video Generators
Zhejiang University of Technology（浙江工业大学）；University of New South Wales (UNSW Sydney)（新南威尔士大学（悉尼））；University of Auckland（奥克兰大学）；Zhejiang University（浙江大学）
Massachusetts Institute of Technology（麻省理工学院）；University of Oxford（牛津大学）；MIT-IBM Watson AI Lab, IBM Research（IBM研究院麻省理工学院-IBM沃森人工智能实验室）；Toyota Research Institute（丰田研究院）；Meta Reality Labs Research（Meta现实实验室）
本文揭示了 SDS 过度平滑的原因在于其高方差噪声采样，并提出在每个更新步骤中通过反演 DDIM 恢复更好的噪声近似，从而显著提升 3D 生成质量且无需额外训练。
Teacher-Guided Student Self-Knowledge Distillation Using Diffusion Model
State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences（人工智能安全国家重点实验室，计算技术研究所，中国科学院）；University of Chinese Academy of Sciences（中国科学院大学）
