Most recently, with the proliferation of IoT devices, computational nodes in manufacturing systems IIoT(Industrial-Internet-of-things) and the lunch of 5G networks, there will be millions of connected devices generating a massive amount of data. In such an environment, the controlling systems need to be intelligent enough to deal with a vast amount of data to detect defects in a real-time process. Driven by such a need, artificial intelligence models such as deep learning have to be deployed into IIoT systems. However, learning and using deep learning models are computationally expensive, so an IoT device with limited computational power could not run such models. To tackle this issue, edge intelligence had emerged as a new paradigm towards running Artificial Intelligence models on edge devices. Although a considerable amount of studies have been proposed in this area, the research is still in the early stages. In this paper, we propose a novel edge-based multi-phase pruning pipelines to ensemble learning on IIoT devices. In the first phase, we generate a diverse ensemble of pruned models, then we apply integer quantisation, next we prune the generated ensemble using a clustering-based technique. Finally, we choose the best representative from each generated cluster to be deployed to a distributed IoT environment. On CIFAR-100 and CIFAR-10, our proposed approach was able to outperform the predictability levels of a baseline model (up to 7%), more importantly, the generated learners have small sizes (up to 90% reduction in the model size) that minimise the required computational capabilities to make an inference on the resource-constraint devices.
Towards General Industrial Intelligence: A Survey of Continual Large Models in Industrial IoT
Shien-Ming Wu School of Intelligent Engineering, South China University of Technology（华南理工大学伍舜德智能工程学院）；School of Mechanical and Automotive Engineering, South China University of Technology（华南理工大学机械与汽车工程学院）；Pazhou Lab（琶洲实验室）；ZJU-UIUC Institute, Zhejiang University（浙江大学ZJU-UIUC学院）；Key Laboratory of Maritime Intelligent Network Information Technology, Ministry of Education, Hohai University（河海大学海洋智能网络信息技术教育部重点实验室）
Network pruning is one of the most dominant methods for reducing the heavy inference cost of deep neural networks. Existing methods often iteratively prune networks to attain high compression ratio without incurring significant loss in performance. However, we argue that conventional methods for retraining pruned networks (i.e., using small, fixed learning rate) are inadequate as they completely ignore the benefits from snapshots of iterative pruning. In this work, we show that strong ensembles can be constructed from snapshots of iterative pruning, which achieve competitive performance and va…
This work presents a novel pipeline that demonstrates what is achievable with a combined effort of state-of-the-art approaches. Specifically, it proposes the novel R2-D2 (Rank twice, reaD twice) pipeline composed of retriever, passage reranker, extractive reader, generative reader and a simple way to combine them. Furthermore, previous work often comes with a massive index of external documents that scales in the order of tens of GiB. This work presents a simple approach for pruning the contents of a massive index such that the open-domain QA system altogether with index, OS, and library compo…
University of Amsterdam（阿姆斯特丹大学）；Canadian Institute for Advanced Research (CIFAR)（加拿大高等研究院（CIFAR））
The success of deep learning in numerous application domains created the de- sire to run and train them on mobile devices. This however, conflicts with their computationally, memory and energy intense nature, leading to a growing interest in compression. Recent work by Han et al. (2015a) propose a pipeline that involves retraining, pruning and quantization of neural network weights, obtaining state-of-the-art compression rates. In this paper, we show that competitive compression rates can be achieved by using a version of soft weight-sharing (Nowlan & Hinton, 1992). Our method achieves both qu…
Equivariant-Aware Structured Pruning for Efficient Edge Deployment: A Comprehensive Framework with Adaptive Fine-Tuning
Faculty of Computers and Information Technology, University of Tabuk（计算机与信息科技学院，塔布克大学）；Artificial Intelligence and Sensing Technologies (AIST) Research Center, University of Tabuk（人工智能与传感技术（AIST）研究中心，塔布克大学）
Texas Tech University（德州理工大学）；The University of Texas at San Antonio（德克萨斯大学圣安东尼奥分校）
Besides accuracy, the model size of convolutional neural networks (CNN) models is another important factor considering limited hardware resources in practical applications. For example, employing deep neural networks on mobile systems requires the design of accurate yet fast CNN for low latency in classification and object detection. To fulfill the need, we aim at obtaining CNN models with both high testing accuracy and small size to address resource constraints in many embedded devices. In particular, this paper focuses on proposing a generic reinforcement learning-based model compression app…
University of Genoa（热那亚大学）；Imperial College London（伦敦帝国理工学院）；University of Sassari（萨萨里大学）
Verification of deep neural networks has witnessed a recent surge of interest, fueled by success stories in diverse domains and by abreast concerns about safety and security in envisaged applications. Complexity and sheer size of such networks are challenging for automated formal verification techniques which, on the other hand, could ease the adoption of deep networks in safety- and security-critical contexts. In this paper we focus on enabling state-of-the-art verification tools to deal with neural networks of some practical interest. We propose a new training pipeline based on network pruni…
HiAP: A Multi-Granular Stochastic Auto-Pruning Framework for Vision Transformers
School of Natural and Computing Sciences, University of Aberdeen（阿伯丁大学自然与计算科学学院）；School of Computing Sciences, University of East Anglia（东安格利亚大学计算科学学院）；Department of Physics and Technology, UiT The Arctic University of Norway（挪威北极大学物理与技术系）
Network pruning is a widely used technique to reduce computation cost and model size for deep neural networks. However, the typical three-stage pipeline, i.e., training, pruning and retraining (fine-tuning) significantly increases the overall training trails. In this paper, we develop a systematic weight-pruning optimization approach based on Surrogate Lagrangian relaxation (SLR), which is tailored to overcome difficulties caused by the discrete nature of the weight-pruning problem while ensuring fast convergence. We further accelerate the convergence of the SLR by using quadratic penalties. M…
Recent years have seen a paradigm shift towards multi-task learning. This calls for memory and energy-efficient solutions for inference in a multi-task scenario. We propose an algorithm-hardware co-design approach called MIME. MIME reuses the weight parameters of a trained parent task and learns task-specific threshold parameters for inference on multiple child tasks. We find that MIME results in highly memory-efficient DRAM storage of neural-network parameters for multiple tasks compared to conventional multi-task inference. In addition, MIME results in input-dependent dynamic neuronal prunin…
University of Washington（华盛顿大学）；New York University（纽约大学）；Unipart Group（优尼帕特集团）
Deep Neural Networks have been used in a wide variety of applications with significant success. However, their highly complex nature owing to comprising millions of parameters has lead to problems during deployment in pipelines with low latency requirements. As a result, it is more desirable to obtain lightweight neural networks which have the same performance during inference time. In this work, we propose a weight based pruning approach in which the weights are pruned gradually based on their momentum of the previous iterations. Each layer of the neural network is assigned an importance valu…
Université de Montréal（蒙特利尔大学）；MILA；IMT Atlantique（IMT大西洋高等矿业学院）；Lab-STICC（STICC实验室）；ISCAS 2019（ISCAS 2019（会议名称，不提取））
Convolutional Neural Networks (CNNs) are state-of-the-art in numerous computer vision tasks such as object classification and detection. However, the large amount of parameters they contain leads to a high computational complexity and strongly limits their usability in budget-constrained devices such as embedded devices. In this paper, we propose a combination of a new pruning technique and a quantization scheme that effectively reduce the complexity and memory usage of convolutional layers of CNNs, and replace the complex convolutional operation by a low-cost multiplexer. We perform experimen…
Constrained Edge AI Deployment: Fine-Tuning vs Distillation for LLM Compression
Department of Computer Science, University of West Florida（计算机科学系，西佛罗里达大学）；Department of Mathematics and Statistics, University of West Florida（数学与统计学系，西佛罗里达大学）；Department of Intelligent Systems and Robotics, University of West Florida（智能系统与机器人系，西佛罗里达大学）；Institute of Human and Machine Cognition（人机认知研究所）；DEVCOM Army Research Laboratory（陆军研究实验室）
Pruning Deep Neural Networks via the Marchenko--Pastur Distribution
Department of Mathematics, Pennsylvania State University（数学系，宾夕法尼亚州立大学）；Department of Computing and Mathematical Sciences, California Institute of Technology（计算与数学科学系，加州理工学院）
Intrusion Detection on Resource-Constrained IoT Devices with Hardware-Aware ML and DL
Faculty of Sciences, Lebanese University（科学学院，黎巴嫩大学）；DITEN, University of Genova（DITEN，热那亚大学）；IMT Atlantique, Lab-STICC UMR CNRS 6285（IMT Atlantique，Lab-STICC UMR CNRS 6285）；School of Arts and Sciences, Lebanese American University（艺术与科学学院，黎巴嫩美国大学）
