The emerging edge-cloud collaborative Deep Learning (DL) paradigm aims at improving the performance of practical DL implementations in terms of cloud bandwidth consumption, response latency, and data privacy preservation. Focusing on bandwidth efficient edge-cloud collaborative training of DNN-based classifiers, we present CDC, a Classification Driven Compression framework that reduces bandwidth consumption while preserving classification accuracy of edge-cloud collaborative DL. Specifically, to reduce bandwidth consumption, for resource-limited edge servers, we develop a lightweight autoencoder with a classification guidance for compression with classification driven feature preservation, which allows edges to only upload the latent code of raw data for accurate global training on the Cloud. Additionally, we design an adjustable quantization scheme adaptively pursuing the tradeoff between bandwidth consumption and classification accuracy under different network conditions, where only fine-tuning is required for rapid compression ratio adjustment. Results of extensive experiments demonstrate that, compared with DNN training with raw data, CDC consumes 14.9 times less bandwidth with an accuracy loss no more than 1.06%, and compared with DNN training with data compressed by AE without guidance, CDC introduces at least 100% lower accuracy loss.
Cross-Domain Collaborative Filtering (CDCF) provides a way to alleviate data sparsity and cold-start problems present in recommendation systems by exploiting the knowledge from related domains. Existing CDCF models are either based on matrix factorization or deep neural networks. Either of the techniques in isolation may result in suboptimal performance for the prediction task. Also, most of the existing models face challenges particularly in handling diversity between domains and learning complex non-linear relationships that exist amongst entities (users/items) within and across domains. In …
RECOD Lab.（RECOD实验室）；WissTek Lab.（WissTek实验室）；School of Electrical and Computing Engineering (FEEC)（电气与计算机工程学院（FEEC））；University of Campinas (UNICAMP)（坎皮纳斯大学（UNICAMP））
In the past few years, mobile deep-learning deployment progressed by leaps and bounds, but solutions still struggle to accommodate its severe and fluctuating operational restrictions, which include bandwidth, latency, computation, and energy. In this work, we help to bridge that gap, introducing the first configurable solution for object detection that manages the triple communication-computation-accuracy trade-off with a single set of weights. Our solution shows state-of-the-art results on COCO-2017, adding only a minor penalty on the base EfficientDet-D2 architecture. Our design is robust to…
WMG, University of Warwick（华威大学制造工程学院）；University of Warwick（华威大学）；Alan Turing Institute（艾伦·图灵研究所）
In artificial multi-agent systems, the ability to learn collaborative policies is predicated upon the agents' communication skills: they must be able to encode the information received from the environment and learn how to share it with other agents as required by the task at hand. We present a deep reinforcement learning approach, Connectivity Driven Communication (CDC), that facilitates the emergence of multi-agent collaborative behaviour only through experience. The agents are modelled as nodes of a weighted graph whose state-dependent edges encode pair-wise messages that can be exchanged. …
Peng Cheng Laboratory（鹏城实验室）；Baidu Inc.（百度公司）；University of Chinese Academy of Sciences（中国科学院大学）；Hong Kong University of Science and Technology（香港科技大学）
The ever-growing model size and scale of compute have attracted increasing interests in training deep learning models over multiple nodes. However, when it comes to training on cloud clusters, especially across remote clusters, huge challenges are faced. In this work, we introduce a general framework, Nebula-I, for collaboratively training deep learning models over remote heterogeneous clusters, the connections between which are low-bandwidth wide area networks (WANs). We took natural language processing (NLP) as an example to show how Nebula-I works in different training phases that include: …
Today, video cameras are deployed in dense for monitoring physical places e.g., city, industrial, or agricultural sites. In the current systems, each camera node sends its feed to a cloud server individually. However, this approach suffers from several hurdles including higher computation cost, large bandwidth requirement for analyzing the enormous data, and privacy concerns. In dense deployment, video nodes typically demonstrate a significant spatio-temporal correlation. To overcome these obstacles in current approaches, this paper introduces CONVINCE, a new approach to look at the network ca…
PWC-MoE: Privacy-Aware Wireless Collaborative Mixture of Experts
Department of Engineering, King’s College London（伦敦国王学院工程系）；Institute for Digital Communication, Friedrich-Alexander-Universität Erlangen-Nürnberg（埃朗根-纽伦堡弗里德里希-亚历山大大学数字通信研究所）
Michigan State University（密歇根州立大学）；Sony AI（索尼人工智能研究院）
As deep learning blooms with growing demand for computation and data resources, outsourcing model training to a powerful cloud server becomes an attractive alternative to training at a low-power and cost-effective end device. Traditional outsourcing requires uploading device data to the cloud server, which can be infeasible in many real-world applications due to the often sensitive nature of the collected data and the limited communication bandwidth. To tackle these challenges, we propose to leverage widely available open-source data, which is a massive dataset collected from public and hetero…
Federated learning becomes a prominent approach when different entities want to learn collaboratively a common model without sharing their training data. However, Federated learning has two main drawbacks. First, it is quite bandwidth inefficient as it involves a lot of message exchanges between the aggregating server and the participating entities. This bandwidth and corresponding processing costs could be prohibitive if the participating entities are, for example, mobile devices. Furthermore, although federated learning improves privacy by not sharing data, recent attacks have shown that it …
Xi’an Jiaotong University（西安交通大学）；Imperial College London（帝国理工学院）
Distributed machine learning (ML) at network edge is a promising paradigm that can preserve both network bandwidth and privacy of data providers. However, heterogeneous and limited computation and communication resources on edge servers (or edges) pose great challenges on distributed ML and formulate a new paradigm of Edge Learning (i.e. edge-cloud collaborative machine learning). In this article, we propose a novel framework of 'learning to learn' for effective Edge Learning (EL) on heterogeneous edges with resource constraints. We first model the dynamic determination of collaboration strate…
National University of Defense Technology（国防科技大学）；Sun Yat-Sen University（中山大学）；Queen Mary University of London（伦敦玛丽女王大学）
Valuable training data is often owned by independent organizations and located in multiple data centers. Most deep learning approaches require to centralize the multi-datacenter data for performance purpose. In practice, however, it is often infeasible to transfer all data to a centralized data center due to not only bandwidth limitation but also the constraints of privacy regulations. Model averaging is a conventional choice for data parallelized training, but its ineffectiveness is claimed by previous studies as deep neural networks are often non-convex. In this paper, we argue that model av…
BiSwift: Bandwidth Orchestrator for Multi-Stream Video Analytics on Edge
State Key Laboratory for Novel Software Technology, Nanjing University（南京大学现代软件工程国家重点实验室）；Institute for AI Industry Research (AIR), Tsinghua University（清华大学人工智能产业研究院）；Institute of Informatik, University of Göttingen（哥廷根大学信息学研究所）
CoCoL: A Communication Efficient Decentralized Collaborative Method for Multi-Robot Systems
College of Control Science and Engineering, Zhejiang University（控制科学与工程学院，浙江大学）；Division of Decision and Control Systems, KTH Royal Institute of Technology（决策与控制系统系，皇家理工学院）
RoboECC: Multi-Factor-Aware Edge-Cloud Collaborative Deployment for VLA Models
School of Computer Science, Peking University, Beijing, China（北京大学计算机科学学院）；School of Computer Science, South China University of Technology, Guangzhou, China（华南理工大学计算机科学学院）；School of Artificial Intelligence, Beijing Normal University, Beijing, China（北京师范大学人工智能学院）；School of EECS, Peking University, Beijing, China（北京大学电子工程与科学学院）
University of Colorado（科罗拉多大学）；Virginia Tech（弗吉尼亚理工学院）
Collaborative deep reinforcement learning (CDRL) algorithms in which multiple agents can coordinate over a wireless network is a promising approach to enable future intelligent and autonomous systems that rely on real-time decision-making in complex dynamic environments. Nonetheless, in practical scenarios, CDRL faces many challenges due to the heterogeneity of agents and their learning tasks, different environments, time constraints of the learning, and resource limitations of wireless networks. To address these challenges, in this paper, a novel semantic-aware CDRL method is proposed to enab…
University of Oulu（奥卢大学）；Centre for Wireless Communications（无线通信中心）
Today's intelligent applications can achieve high performance accuracy using machine learning (ML) techniques, such as deep neural networks (DNNs). Traditionally, in a remote DNN inference problem, an edge device transmits raw data to a remote node that performs the inference task. However, this may incur high transmission energy costs and puts data privacy at risk. In this paper, we propose a technique to reduce the total energy bill at the edge device by utilizing model compression and time-varying model split between the edge and remote nodes. The time-varying representation accounts for ti…
NSC-SL: A Bandwidth-Aware Neural Subspace Compression for Communication-Efficient Split Learning
School of Computer and Information Engineering, Xiamen University of Technology, Xiamen, China（厦门理工学院计算机与信息工程学院）；Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong, China（香港大学电子与电气工程系）；Department of Computer Science, City University of Hong Kong, Hong Kong, China（香港城市大学计算机科学系）；Department of Computer Science, The University of Hong Kong, Hong Kong, China（香港大学计算机科学系）；School of Computing, National University of Singapore, Singapore（新加坡国立大学计算机学院）
Galaxy: A Resource-Efficient Collaborative Edge AI System for In-situ Transformer Inference
School of Computer Science and Engineering, Sun Yat-sen University（中山大学计算机科学与工程学院）；IoT Thrust and Research Center for Digital World with Intelligent Things, HKUST (Guangzhou)（香港科技大学（广州）智能物联数字世界研究中心及物联网学科）；Data Science and Analytics Thrust, HKUST (Guangzhou)（香港科技大学（广州）数据科学与分析学科）；Department of Computer Science and Engineering, HKUST（香港科技大学计算机科学与工程系）
Spectrum sharing among users is a fundamental problem in the management of any wireless network. In this paper, we discuss the problem of distributed spectrum collaboration without central management under general unknown channels. Since the cost of communication, coordination and control is rapidly increasing with the number of devices and the expanding bandwidth used there is an obvious need to develop distributed techniques for spectrum collaboration where no explicit signaling is used. In this paper, we combine game-theoretic insights with deep Q-learning to provide a novel asymptotically …
The integration of edge computing in next-generation mobile networks is bringing low-latency and high-bandwidth ubiquitous connectivity to a myriad of cyber-physical systems. This will further boost the increasing intelligence that is being embedded at the edge in various types of autonomous systems, where collaborative machine learning has the potential to play a significant role. This paper discusses some of the challenges in multi-agent distributed deep reinforcement learning that can occur in the presence of byzantine or malfunctioning agents. As the simulation-to-reality gap gets bridged,…
