arXiv 2108.089102021-08eess.IVcs.AIcs.CVcs.LGcs.NE

Achieving on-Mobile Real-Time Super-Resolution with Neural Architecture and Pruning Search

Northeastern University（东北大学）；College of William & Mary（威廉与玛丽学院）；Cleveland State University（克利夫兰州立大学）

Zheng Zhan, Yifan Gong, Pu Zhao, Geng Yuan, Wei Niu, Yushu Wu, Tianyun Zhang, Malith Jayaweera, David Kaeli, Bin Ren, Xue Lin, Yanzhi Wang

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

英文摘要相关论文

英文摘要：

Though recent years have witnessed remarkable progress in single image super-resolution (SISR) tasks with the prosperous development of deep neural networks (DNNs), the deep learning methods are confronted with the computation and memory consumption issues in practice, especially for resource-limited platforms such as mobile devices. To overcome the challenge and facilitate the real-time deployment of SISR tasks on mobile, we combine neural architecture search with pruning search and propose an automatic search framework that derives sparse super-resolution (SR) models with high image quality while satisfying the real-time inference requirement. To decrease the search cost, we leverage the weight sharing strategy by introducing a supernet and decouple the search problem into three stages, including supernet construction, compiler-aware architecture and pruning search, and compiler-aware pruning ratio search. With the proposed framework, we are the first to achieve real-time SR inference (with only tens of milliseconds per frame) for implementing 720p resolution with competitive image quality (in terms of PSNR and SSIM) on mobile platforms (Samsung Galaxy S20).

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

Good Subnetworks Provably Exist: Pruning via Greedy Forward Selection

the University of Texas, Austin（得克萨斯大学奥斯汀分校）；the University of Chicago（芝加哥大学）；Google Research（谷歌研究院）

Recent empirical works show that large deep neural networks are often highly redundant and one can find much smaller subnetworks without a significant drop of accuracy. However, most existing methods of network pruning are empirical and heuristic, leaving it open whether good subnetworks provably exist, how to find them efficiently, and if network pruning can be provably better than direct training using gradient descent. We answer these problems positively by proposing a simple greedy selection approach for finding good subnetworks, which starts from an empty network and greedily adds importa…

2020-10-19cs.LGstat.ML

Dynamic Runtime Feature Map Pruning

High bandwidth requirements are an obstacle for accelerating the training and inference of deep neural networks. Most previous research focuses on reducing the size of kernel maps for inference. We analyze parameter sparsity of six popular convolutional neural networks - AlexNet, MobileNet, ResNet-50, SqueezeNet, TinyNet, and VGG16. Of the networks considered, those using ReLU (AlexNet, SqueezeNet, VGG16) contain a high percentage of 0-valued parameters and can be statically pruned. Networks with Non-ReLU activation functions in some cases may not contain any 0-valued parameters (ResNet-50, Ti…

2019-04-02cs.CVcs.LG

Enable Deep Learning on Mobile Devices: Methods, Systems, and Applications

Massachusetts Institute of Technology（麻省理工学院）

Deep neural networks (DNNs) have achieved unprecedented success in the field of artificial intelligence (AI), including computer vision, natural language processing and speech recognition. However, their superior performance comes at the considerable cost of computational complexity, which greatly hinders their applications in many resource-constrained devices, such as mobile phones and Internet of Things (IoT) devices. Therefore, methods and techniques that are able to lift the efficiency bottleneck while preserving the high accuracy of DNNs are in great demand in order to enable numerous edg…

2022-04-25cs.CLcs.CVcs.LG

Task-Specific Expert Pruning for Sparse Mixture-of-Experts

Beihang University（北京航空航天大学）；The Institute of Acoustics of the Chinese Academy of Sciences（中国科学院声学研究所）；Microsoft（微软）

The sparse Mixture-of-Experts (MoE) model is powerful for large-scale pre-training and has achieved promising results due to its model capacity. However, with trillions of parameters, MoE is hard to be deployed on cloud or mobile environment. The inference of MoE requires expert parallelism, which is not hardware-friendly and communication expensive. Especially for resource-limited downstream tasks, such sparse structure has to sacrifice a lot of computing efficiency for limited performance gains. In this work, we observe most experts contribute scarcely little to the MoE fine-tuning and infer…

2022-06-02cs.AIcs.LG

Fire Together Wire Together: A Dynamic Pruning Approach with Self-Supervised Mask Prediction

University of Alberta（阿尔伯塔大学）；Huawei（华为）

Dynamic model pruning is a recent direction that allows for the inference of a different sub-network for each input sample during deployment. However, current dynamic methods rely on learning a continuous channel gating through regularization by inducing sparsity loss. This formulation introduces complexity in balancing different losses (e.g task loss, regularization loss). In addition, regularization based methods lack transparent tradeoff hyperparameter selection to realize a computational budget. Our contribution is two-fold: 1) decoupled task and pruning losses. 2) Simple hyperparameter se…

2022-06-29cs.CVcs.LG

基于同态熵增的预训练模型个性化压缩算法研究

Research on Personalized Compression Algorithm for Pre-trained Models Based on Homomorphic Entropy Increase

Anhui University（安徽大学）；Beihang University（北京航空航天大学）

针对预训练模型部署难问题，提出基于同态熵增的分层剪枝策略，结合压缩感知与步长缓冲机制，在保障精度的同时显著压缩模型，助力移动端个性化AI部署。

2024-08-16cs.LG

更多相关论文收起更多论文

DDPNAS: Efficient Neural Architecture Search via Dynamic Distribution Pruning

Xiamen University（厦门大学）；Peng Cheng Laboratory（鹏城实验室）；Samsara（Samsara公司）；Beihang University（北京航空航天大学）；Tencent Co., Ltd.（腾讯科技有限公司）；Terminus Group（特斯联集团）

Neural Architecture Search (NAS) has demonstrated state-of-the-art performance on various computer vision tasks. Despite the superior performance achieved, the efficiency and generality of existing methods are highly valued due to their high computational complexity and low generality. In this paper, we propose an efficient and unified NAS framework termed DDPNAS via dynamic distribution pruning, facilitating a theoretical bound on accuracy and efficiency. In particular, we first sample architectures from a joint categorical distribution. Then the search space is dynamically pruned and its dis…

2023-03-10cs.CVcs.LG

Sparsity Meets Robustness: Channel Pruning for the Feynman-Kac Formalism Principled Robust Deep Neural Nets

University of California, Irvine（加州大学欧文分校）；University of California, Los Angeles（加州大学洛杉矶分校）

Deep neural nets (DNNs) compression is crucial for adaptation to mobile devices. Though many successful algorithms exist to compress naturally trained DNNs, developing efficient and stable compression algorithms for robustly trained DNNs remains widely open. In this paper, we focus on a co-design of efficient DNN compression algorithms and sparse neural architectures for robust and accurate deep learning. Such a co-design enables us to advance the goal of accommodating both sparsity and robustness. With this objective in mind, we leverage the relaxed augmented Lagrangian based algorithms to pr…

2020-03-02cs.AIcs.LGstat.ML

PocketSR：口袋手机中的超分辨率专家

PocketSR: The Super-Resolution Expert in Your Pocket Mobiles

Tsinghua University（清华大学）；Joy Future Academy（京东探索研究院）；HKUST (GZ)（香港科技大学（广州））

针对真实世界图像超分辨率在移动端部署的高延迟问题，本文提出超轻量单步模型PocketSR，通过LiteED、在线退火剪枝和多层特征蒸馏实现146M参数、0.8秒处理4K图像，并达到先进性能。

2025-10-03cs.CV

RT3D: Achieving Real-Time Execution of 3D Convolutional Neural Networks on Mobile Devices

William & Mary（威廉玛丽学院）；Northeastern University（东北大学）；North Carolina State University（北卡罗来纳州立大学）；MIT-IBM Watson AI Lab（麻省理工学院-IBM沃森人工智能实验室）

Mobile devices are becoming an important carrier for deep learning tasks, as they are being equipped with powerful, high-end mobile CPUs and GPUs. However, it is still a challenging task to execute 3D Convolutional Neural Networks (CNNs) targeting for real-time performance, besides high inference accuracy. The reason is more complex model structure and higher model dimensionality overwhelm the available computation/storage resources on mobile devices. A natural way may be turning to deep learning weight pruning techniques. However, the direct generalization of existing 2D CNN weight pruning me…

2021-01-03cs.CVcs.LGcs.NE

An Image Enhancing Pattern-based Sparsity for Real-time Inference on Mobile Devices

Northeastern University（东北大学）；College of William and Mary（威廉与玛丽学院）；Syracuse University（雪城大学）；IBM Research（IBM研究院）；Lehigh University（理海大学）；George Mason University（乔治梅森大学）；DiDi AI Labs（滴滴AI Labs）；Tsinghua University（清华大学）

Weight pruning has been widely acknowledged as a straightforward and effective method to eliminate redundancy in Deep Neural Networks (DNN), thereby achieving acceleration on various platforms. However, most of the pruning techniques are essentially trade-offs between model accuracy and regularity which lead to impaired inference accuracy and limited on-device acceleration performance. To solve the problem, we introduce a new sparsity dimension, namely pattern-based sparsity that comprises pattern and connectivity sparsity, and becoming both highly accurate and hardware friendly. With carefull…

2020-07-05cs.CVcs.LGcs.NE

Cluster Pruning: An Efficient Filter Pruning Method for Edge AI Vision Applications

Singapore University of Technology and Design（新加坡科技设计大学）

Even though the Convolutional Neural Networks (CNN) has shown superior results in the field of computer vision, it is still a challenging task to implement computer vision algorithms in real-time at the edge, especially using a low-cost IoT device due to high memory consumption and computation complexities in a CNN. Network compression methodologies such as weight pruning, filter pruning, and quantization are used to overcome the above mentioned problem. Even though filter pruning methodology has shown better performances compared to other techniques, irregularity of the number of filters prun…

2020-03-05cs.CVcs.LGeess.IV