The training of deep and/or convolutional neural networks (DNNs/CNNs) is traditionally done on servers with powerful CPUs and GPUs. Recent efforts have emerged to localize machine learning tasks fully on the edge. This brings advantages in reduced latency and increased privacy, but necessitates working with resource-constrained devices. Approaches for inference and training in mobile and edge devices based on pruning, quantization or incremental and transfer learning require trading off accuracy. Several works have explored distributing inference operations on mobile and edge clusters instead. However, there is limited literature on distributed training on the edge. Existing approaches all require a central, potentially powerful edge or cloud server for coordination or offloading. In this paper, we describe an approach for distributed CNN training exclusively on mobile and edge devices. Our approach is beneficial for the initial CNN layers that are feature map dominated. It is based on partitioning forward inference and back-propagation operations among devices through tiling and fusing to maximize locality and expose communication and memory-aware parallelism. We also introduce the concept of layer grouping to further fine-tune performance based on computation and communication trade-off. Results show that for a cluster of 2-6 quad-core Raspberry Pi3 devices, training of an object-detection CNN provides a 2x-15x speedup with respect to a single core and up to 8x reduction in memory usage per device, all without sacrificing accuracy. Grouping offers up to 1.5x speedup depending on the reference profile and batch size.
Convolutional neural networks have witnessed remarkable improvements in computational efficiency in recent years. A key driving force has been the idea of trading-off model expressivity and efficiency through a combination of $1\times 1$ and depth-wise separable convolutions in lieu of a standard convolutional layer. The price of the efficiency, however, is the sub-optimal flow of information across space and channels in the network. To overcome this limitation, we present MUXConv, a layer that is designed to increase the flow of information by progressively multiplexing channel and spatial in…
The ubiquity of mobile devices has led to the proliferation of mobile services that provide personalized and context-aware content to their users. Modern mobile services are distributed between end-devices, such as smartphones, and remote servers that reside in the cloud. Such services thrive on their ability to predict future contexts to pre-fetch content or make context-specific recommendations. An increasingly common method to predict future contexts, such as location, is via machine learning (ML) models. Recent work in context prediction has focused on ML model personalization where a pers…
Cloud computing is the way by which we connect to servers, large systems into a distributed secure manner without worrying about local memory limits. Here this paper we proposed a Novel Distributed Database Architectural Model for Mobile Cloud Computing (NDDAMMCC). Due to the exponential growth of wireless technologies and internet which are following Nielsen's Law of Internet Bandwidth, we are in the new era of cloud computing. In the recent technological era, smart mobile devices play a big role in all sort of day-by-day human needs. The applicability is so huge that the number of apps insta…
Chalmers University of Technology（查尔姆斯理工大学）；University of Southern California（南加州大学）
Modern machine learning techniques are successfully being adapted to data modeled as graphs. However, many real-world graphs are typically very large and do not fit in memory, often making the problem of training machine learning models on them intractable. Distributed training has been successfully employed to alleviate memory problems and speed up training in machine learning domains in which the input data is assumed to be independently identical distributed (i.i.d). However, distributing the training of non i.i.d data such as graphs that are used as training inputs in Graph Convolutional N…
Currently, various hardware and software companies are developing augmented reality devices, most prominently Microsoft with its Hololens. Besides gaming, such devices can be used for serious pervasive applications, like interactive mobile simulations to support engineers in the field. Interactive simulations have high demands on resources, which the mobile device alone is unable to satisfy. Therefore, we propose a framework to support mobile simulations by distributing the computation between the mobile device and a remote server based on the reduced basis method. Evaluations show that we can…
Old Dominion University（欧道明大学）；Stony Brook University（纽约州立大学石溪分校）
Originated from distributed learning, federated learning enables privacy-preserved collaboration on a new abstracted level by sharing the model parameters only. While the current research mainly focuses on optimizing learning algorithms and minimizing communication overhead left by distributed learning, there is still a considerable gap when it comes to the real implementation on mobile devices. In this paper, we start with an empirical experiment to demonstrate computation heterogeneity is a more pronounced bottleneck than communication on the current generation of battery-powered mobile devi…
At present, the de-facto standard for providing contents in the Internet is the World Wide Web. A technology, which is now emerging on the Web, is Content-Based Image Retrieval (CBIR). CBIR applies methods and algorithms from computer science to analyse and index images based on their visual content. Mobile agents push the flexibility of distributed systems to their limits since not only computations are dynamically distributed but also the code that performs them. The current commercial applet-based methodologies for accessing image database systems offer limited flexibility, scalability and …
Advances in mobile computing have paved the way for new types of distributed applications that can be executed solely by mobile devices on device-to-device (D2D) ecosystems (e.g., crowdsensing). Sophisticated applications, like cryptocurrencies, need distributed ledgers to function. Distributed ledgers, such as blockchains and directed acyclic graphs (DAGs), employ consensus protocols to add data in the form of blocks. However, such protocols are designed for resourceful devices that are interconnected via the Internet. Moreover, existing distributed ledgers are not deployable to D2D ecosystem…
We present Blizzard, a Byzantine Fault Tolerant (BFT) distributed ledger protocol that is aimed at making mobile devices first-class citizens in the consensus process. Blizzard introduces a novel two-tier architecture by having the mobile nodes communicate through online brokers, and includes a decentralized matching scheme to ensure each node connects to a certain number of random brokers. Through mathematical analysis, we derive a guaranteed safety region (i.e. the set of ratios of malicious nodes and malicious brokers for which the safety is assured) for the Blizzard protocol. Liveness is s…
Distributed Services Architecture with support for mobile agents between services, offer significantly improved communication and computational flexibility. The uses of agents allow execution of complex operations that involve large amounts of data to be processed effectively using distributed resources. The prototype system Distributed Agents for Mobile and Dynamic Services (DIAMOnDS), allows a service to send agents on its behalf, to other services, to perform data manipulation and processing. Agents have been implemented as mobile services that are discovered using the Jini Lookup mechanism…
Sharif University of Technology（谢里夫理工大学）；University of California, Davis（加利福尼亚大学戴维斯分校）
Many mobile applications running on smartphones and wearable devices would potentially benefit from the accuracy and scalability of deep CNN-based machine learning algorithms. However, performance and energy consumption limitations make the execution of such computationally intensive algorithms on mobile devices prohibitive. We present a GPU-accelerated library, dubbed CNNdroid, for execution of trained deep CNNs on Android-based mobile devices. Empirical evaluations show that CNNdroid achieves up to 60X speedup and 130X energy saving on current mobile devices. The CNNdroid open source library…
We propose a Bayesian method for distributed sequential localization of mobile networks composed of both cooperative agents and noncooperative objects. Our method provides a consistent combination of cooperative self-localization (CS) and distributed tracking (DT). Multiple mobile agents and objects are localized and tracked using measurements between agents and objects and between agents. For a distributed operation and low complexity, we combine particle-based belief propagation with a consensus or gossip scheme. High localization accuracy is achieved through a probabilistic information tran…
Following the wide investigation in distributed computing issues by mobile entities of the last two decades, we consider the need of a structured methodology to tackle the arisen problems. The aim is to simplify both the design of the resolution algorithms and the writing of the required correctness proofs. We would encourage the usage of a common framework in order to help both algorithm designer and reviewers in the intricate work of analyzing the proposed resolution strategies. In order to better understand the potentials of our methodology, we consider the Pattern Formation (PF) problem ap…
Coordinated checkpointing is an effective fault tolerant technique in distributed system as it avoids the domino effect and require minimum storage requirement. Most of the earlier coordinated checkpoint algorithms block their computation during checkpointing and forces minimum-process or non-blocking but forces all nodes to takes checkpoint even though many of them may not be necessary or non-blocking minimum-process but takes useless checkpoints or reduced useless checkpoint but has higher synchronization message overhead or has high checkpoint request propagation time. Hence in mobile distr…
The popularity of neural networks (NNs) spans academia, industry, and popular culture. In particular, convolutional neural networks (CNNs) have been applied to many image based machine learning tasks and have yielded strong results. The availability of hardware/software systems for efficient training and deployment of large and/or deep CNN models has been, and continues to be, an important consideration for the field. Early systems for NN computation focused on leveraging existing dense linear algebra techniques and libraries. Current approaches use low-level machine specific programming and/o…
INRIA（法国国家信息与自动化研究所）；Université Paris Saclay（巴黎萨克雷大学）
Convolutional dictionary learning (CDL) estimates shift invariant basis adapted to multidimensional data. CDL has proven useful for image denoising or inpainting, as well as for pattern discovery on multivariate signals. As estimated patterns can be positioned anywhere in signals or images, optimization techniques face the difficulty of working in extremely high dimensions with millions of pixels or time samples, contrarily to standard patch-based dictionary learning. To address this optimization problem, this work proposes a distributed and asynchronous algorithm, employing locally greedy coo…
In this paper we focus on Internet-based simulation, a form of distributed simulation in which a set of execution units that are physically located around the globe work together to run a simulation model. This setup is very challenging because of the latency/variability of communications. Thus, clever mechanisms must be adopted in the distributed simulation, such as the adaptive partitioning of the simulated model and load balancing strategies among execution units. We simulate a wireless model over a real Internet-based distributed simulation setup, and evaluate the scalability of the simula…
Several efficient distributed algorithms have been developed for matrix-matrix multiplication: the 3D algorithm, the 2D SUMMA algorithm, and the 2.5D algorithm. Each of these algorithms was independently conceived and they trade-off memory needed per node and the inter-node data communication volume. The convolutional neural network (CNN) computation may be viewed as a generalization of matrix-multiplication combined with neighborhood stencil computations. We develop communication-efficient distributed-memory algorithms for CNNs that are analogous to the 2D/2.5D/3D algorithms for matrix-matrix…
Mobile sensor networks (MSNs) have emerged from the interaction between mobile robotics and wireless sensor networks. MSNs can be deployed in harsh environments, where failures in some nodes can partition MSNs into disconnected network segments or reduce the coverage area. A k-connected network can tolerate at least k-1 arbitrary node failures without losing its connectivity. In this study, we present a coverage-aware distributed k-connectivity maintenance (restoration) algorithm that generates minimum-cost movements of active nodes after a node failure to preserve a persistent k value subject…
Following the recent success of deep neural networks (DNN) on video computer vision tasks, performing DNN inferences on videos that originate from mobile devices has gained practical significance. As such, previous approaches developed methods to offload DNN inference computations for images to cloud servers to manage the resource constraints of mobile devices. However, when it comes to video data, communicating information of every frame consumes excessive network bandwidth and renders the entire system susceptible to adverse network conditions such as congestion. Thus, in this work, we seek …
Mobile edge computing (MEC) and next-generation mobile networks are set to disrupt the way intelligent and autonomous systems are interconnected. This will have an effect on a wide range of domains, from the Internet of Things to autonomous mobile robots. The integration of such a variety of MEC services in a inherently distributed architecture requires a robust system for managing hardware resources, balancing the network load and securing the distributed applications. Blockchain technology has emerged a solution for managing MEC services, with consensus protocols and data integrity checks th…
Mobile clients that consume and produce data are abundant in fog environments and low latency access to this data can only be achieved by storing it in their close physical proximity. To adapt data replication in fog data stores in an efficient manner and make client data available at the fog node that is closest to the client, the systems need to predict both client movement and pauses in data consumption. In this paper, we present variations of Markov model algorithms that can run on clients to increase the data availability while minimizing excess data. In a simulation, we find the availabi…
Univ. of Bremen（不来梅大学）；Institute for Digitization（数字化研究所）
Agent-based modelling (ABM), simulation (ABS), and distributed computation (ABC) are established methods. The Internet and Web-based technologies are suitable carriers. This paper is a technical report with some tutorial aspects of the JavaScript Agent Machine (JAM) platform and the programming of agents with AgentJS, a sub-set of the widely used JavaScript programming language for the programming of mobile state-based reactive agents. In addition to explaining the motivation for particular design choices and introducing core concepts of the architecture and the programming of agents in JavaSc…
Predictive analytics in Mobile Edge Computing (MEC) based Internet of Things (IoT) is becoming a high demand in many real-world applications. A prediction problem in an MEC-based IoT environment typically corresponds to a collection of tasks with each task solved in a specific MEC environment based on the data accumulated locally, which can be regarded as a Multi-task Learning (MTL) problem. However, the heterogeneity of the data (non-IIDness) accumulated across different MEC environments challenges the application of general MTL techniques in such a setting. Federated MTL (FMTL) has recently …
In this work we consider the problem of gathering autonomous robots in the plane. In particular, we consider non-transparent unit-disc robots (i.e., fat) in an asynchronous setting. Vision is the only mean of coordination. Using a state-machine representation we formulate the gathering problem and develop a distributed algorithm that solves the problem for any number of robots. The main idea behind our algorithm is for the robots to reach a configuration in which all the following hold: (a) The robots' centers form a convex hull in which all robots are on the convex, (b) Each robot can see all…
In Distributed Interactive Applications (DIA) such as multiplayer games, where many participants are involved in a same game session and communicate through a network, they may have an inconsistent view of the virtual world because of the communication delays across the network. This issue becomes even more challenging when communicating through a cellular network while executing the DIA client on a mobile terminal. Consistency maintenance algorithms may be used to obtain a uniform view of the virtual world. These algorithms are very complex and hard to program and therefore, the implementatio…
In this paper we consider the problem of high accuracy localization of mobile nodes in a multipath-rich environment where sub-meter accuracies are required. We employ a peer to peer framework where the vehicles/nodes can get pairwise multipath-degraded ranging estimates in local neighborhoods together with a fixed number of anchor nodes. The challenge is to overcome the multipath-barrier with redundancy in order to provide the desired accuracies especially under severe multipath conditions when the fraction of received signals corrupted by multipath is dominating. We invoke a message passing a…
In Covid-19 pandemic, the number of users connecting to the Internet using mobile devices increased. People are doing there every task using mobile phones [16]. These devices are battery-powered and have limited computation capabilities. Their computational capabilities can be enhanced by computation offloading in which required computation is to be done on a third-party server on a cloud instead of the device itself. The cloud offers virtually infinite computation and storage. We proposed that by exploiting parallelism within an application call hierarchy we can decrease the execution time of…
Developing personal robots that can perform a diverse range of manipulation tasks in unstructured environments necessitates solving several challenges for robotic grasping systems. We take a step towards this broader goal by presenting the first RL-based system, to our knowledge, for a mobile manipulator that can (a) achieve targeted grasping generalizing to unseen target objects, (b) learn complex grasping strategies for cluttered scenes with occluded objects, and (c) perform active vision through its movable wrist camera to better locate objects. The system is informed of the desired target …
The increased mobile connectivity, the range and number of services available in various computing environments in the network, demand mobile applications to be highly dynamic to be able to efficiently incorporate those services into applications, along with other local capabilities on mobile devices. However, the monolithic structure and mostly static configuration of mobile application components today limit application's ability to dynamically manage internal components, to be able to adapt to the user and the environment, and utilize various services in the network for improving the applic…
Training a machine learning model is both compute and data-intensive. Most of the model training is performed on high performance compute nodes and the training data is stored near these nodes for faster training. But there is a growing interest in enabling training near the data. For instance, mobile devices are rich sources of training data. It may not be feasible to consolidate the data from mobile devices into a cloud service, due to bandwidth and data privacy reasons. Training at mobile devices is however fraught with challenges. First mobile devices may join or leave the distributed sett…
Recent advances in cloud computing have simplified the way that both software development and testing are performed. Unfortunately, this is not true for battery testing for which state of the art test-beds simply consist of one phone attached to a power meter. These test-beds have limited resources, access, and are overall hard to maintain; for these reasons, they often sit idle with no experiment to run. In this paper, we propose to share existing battery testing setups and build BatteryLab, a distributed platform for battery measurements. Our vision is to transform independent battery testin…
Consider a set of $n$ simple autonomous mobile robots (asynchronous, no common coordinate system, no identities, no central coordination, no direct communication, no memory of the past, non-rigid, deterministic) initially in distinct locations, moving freely in the plane and able to sense the positions of the other robots. We study the primitive task of the robots arranging themselves on the vertices of a regular $n$-gon not fixed in advance (Uniform Circle Formation). In the literature, the existing algorithmic contributions are limited to conveniently restricted sets of initial configuration…
