Recent studies have shown that collaborative intelligence (CI) is a promising framework for deployment of Artificial Intelligence (AI)-based services on mobile devices. In CI, a deep neural network is split between the mobile device and the cloud. Deep features obtained at the mobile are compressed and transferred to the cloud to complete the inference. So far, the methods in the literature focused on transferring a single deep feature tensor from the mobile to the cloud. Such methods are not applicable to some recent, high-performance networks with multiple branches and skip connections. In this paper, we propose the first bit allocation method for multi-stream, multi-task CI. We first establish a model for the joint distortion of the multiple tasks as a function of the bit rates assigned to different deep feature tensors. Then, using the proposed model, we solve the rate-distortion optimization problem under a total rate constraint to obtain the best rate allocation among the tensors to be transferred. Experimental results illustrate the efficacy of the proposed scheme compared to several alternative bit allocation methods.
University of Science and Technology of China（中国科学技术大学）
Rapid growing intelligent applications require optimized bit allocation in image/video coding to support specific task-driven scenarios such as detection, classification, segmentation, etc. Some learning-based frameworks have been proposed for this purpose due to their inherent end-to-end optimization mechanisms. However, it is still quite challenging to integrate these task-driven metrics seamlessly into traditional hybrid coding framework. To the best of our knowledge, this paper is the first work trying to solve this challenge based on reinforcement learning (RL) approach. Specifically, we …
In recent studies, collaborative intelligence (CI) has emerged as a promising framework for deployment of Artificial Intelligence (AI)-based services on mobile/edge devices. In CI, the AI model (a deep neural network) is split between the edge and the cloud, and intermediate features are sent from the edge sub-model to the cloud sub-model. In this paper, we study bit allocation for feature coding in multi-stream CI systems. We model task distortion as a function of rate using convex surfaces similar to those found in distortion-rate theory. Using such models, we are able to provide closed-form…
National Yang Ming Chiao Tung University（国立阳明交通大学）
This paper presents a reinforcement learning (RL) framework that leverages Frank-Wolfe policy optimization to address frame-level bit allocation for HEVC/H.265. Most previous RL-based approaches adopt the single-critic design, which weights the rewards for distortion minimization and rate regularization by an empirically chosen hyper-parameter. More recently, the dual-critic design is proposed to update the actor network by alternating the rate and distortion critics. However, the convergence of training is not guaranteed. To address this issue, we introduce Neural Frank-Wolfe Policy Optimizat…
Brown University（布朗大学）；Ben-Gurion University（本-古里安大学）
Quantized neural networks (QNNs) are among the main approaches for deploying deep neural networks on low resource edge devices. Training QNNs using different levels of precision throughout the network (dynamic quantization) typically achieves superior trade-offs between performance and computational load. However, optimizing the different precision levels of QNNs can be complicated, as the values of the bit allocations are discrete and difficult to differentiate for. Also, adequately accounting for the dependencies between the bit allocation of different layers is not straight-forward. To meet…
AlphaQ: Calibration-Free Bit Allocation for Mixture-of-Experts Quantization
Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；ELLIS Institute Tübingen（图宾根ELLIS研究所）；Tübingen AI Center（图宾根人工智能中心）；Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Xiamen University（厦门大学多媒体可信感知与高效计算重点实验室）；International Computer Science Institute（国际计算机科学研究所）；Lawrence Berkeley National Laboratory（伯克利国家实验室）；University of California, Berkeley（加州大学伯克利分校）；Liquid AI
BitsMoE: Cost-Aware Bit Allocation in Spectral Space for MoE LLM Quantization
School of Microelectronics, University of Science and Technology of China（中国科学技术大学微电子学院）；College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算与数据科学学院）；School of Electrical and Electronic Engineering, Nanyang Technological University（南洋理工大学电子与电气工程学院）
Joint Optimization of Model Partitioning and Resource Allocation for Anti-Jamming Collaborative Inference Systems
College of Information Engineering, Zhejiang University of Technology（浙江工业大学信息工程学院）；School of Communications and Information Engineering, Nanjing University of Posts and Telecommunications（南京邮电大学通信与信息工程学院）；Department of Electronic and Electrical Engineering, University College London（伦敦大学学院电子与电气工程系）；Department of Electronic Engineering, Kyung Hee University（庆熙大学电子工程系）；Department of Electronics and Information Convergence Engineering, Kyung Hee University（庆熙大学电子信息融合工程系）
When reasoning about tasks that involve large amounts of data, a common approach is to represent data items as objects in the Hamming space where operations can be done efficiently and effectively. Object similarity can then be computed by learning binary representations (hash codes) of the objects and computing their Hamming distance. While this is highly efficient, each bit dimension is equally weighted, which means that potentially discriminative information of the data is lost. A more expressive alternative is to use real-valued vector representations and compute their inner product; this …
Towards Efficient and Accurate Spiking Neural Networks via Adaptive Bit Allocation
The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences（认知与决策智能复杂系统重点实验室，自动化研究所，中国科学院）；School of Future Technology, University of Chinese Academy of Sciences（未来技术学院，中国科学院大学）；China Electric Power Research Institute Co., Ltd（中国电力科学研究院有限公司）
Queen Mary University of London（伦敦玛丽女王大学）；British Broadcasting Corporation（英国广播公司）
Rate-control is essential to ensure efficient video delivery. Typical rate-control algorithms rely on bit allocation strategies, to appropriately distribute bits among frames. As reference frames are essential for exploiting temporal redundancies, intra frames are usually assigned a larger portion of the available bits. In this paper, an accurate method to estimate number of bits and quality of intra frames is proposed, which can be used for bit allocation in a rate-control scheme. The algorithm is based on deep learning, where networks are trained using the original frames as inputs, while di…
Mobile Edge Computing (MEC) has emerged as a promising supporting architecture providing a variety of resources to the network edge, thus acting as an enabler for edge intelligence services empowering massive mobile and Internet of Things (IoT) devices with AI capability. With the assistance of edge servers, user equipments (UEs) are able to run deep neural network (DNN) based AI applications, which are generally resource-hungry and compute-intensive, such that an individual UE can hardly afford by itself in real time. However the resources in each individual edge server are typically limited.…
In collaborative intelligence applications, part of a deep neural network (DNN) is deployed on a relatively low-complexity device such as a mobile phone or edge device, and the remainder of the DNN is processed where more computing resources are available, such as in the cloud. This paper presents a novel lightweight compression technique designed specifically to code the activations of a split DNN layer, while having a low complexity suitable for edge devices and not requiring any retraining. We also present a modified entropy-constrained quantizer design algorithm optimized for clipped activ…
Timeliness-Oriented Scheduling and Resource Allocation in Multi-Region Collaborative Perception
School of Electronic and Information Engineering, Beijing Jiaotong University（北京交通大学电子与信息工程学院）；Beijing National Research Center for Information Science and Technology, Department of Electronic Engineering, Tsinghua University（北京信息科学与技术国家研究中心，清华大学电子工程系）
Task-Aware QUBO Allocation for Mixed-Precision Quantization
Innopolis University（因诺波利斯大学）；Q Deep；Moscow Independent Research Institute of Artificial Intelligence (MIRAI)（莫斯科独立人工智能研究院（MIRAI））；Sirius University of Science and Technology（天狼星科技大学）
