Virtual clusters are widely used computing platforms than can be deployed in multiple cloud platforms. The ability to dynamically grow and shrink the number of nodes has paved the way for customised elastic computing both for High Performance Computing and High Throughput Computing workloads. However, elasticity is typically restricted to a single cloud site, thus hindering the ability to provision computational resources from multiple geographically distributed cloud sites. To this aim, this paper introduces an architecture of open-source components that coherently deploy a virtual elastic cluster across multiple cloud sites to perform large-scale computing. These hybrid virtual elastic clusters are automatically deployed and configured using an Infrastructure as Code (IaC) approach on a distributed hybrid testbed that spans different organizations, including on-premises and public clouds, supporting automated tunneling of communications across the cluster nodes with advanced VPN topologies. The results indicate that cluster-based computing of embarrassingly parallel jobs can benefit from hybrid virtual clusters that aggregate computing resources from multiple cloud back-ends and bring them together into a dedicated, albeit virtual network.
The work presented in this article has been partially funded by the European Union's (EU) Horizon 2020 research project DEEP Hybrid-DataCloud (grant agreement No 777435).
Cloud computing datacenters provide millions of virtual machines in actual cloud markets. In this context, Virtual Machine Placement (VMP) is one of the most challenging problems in cloud infrastructure management, considering the large number of possible optimization criteria and different formulations that could be studied. Considering the on-demand model of cloud computing, the VMP problem should be solved dynamically to efficiently attend typical workload of modern applications. This work proposes a taxonomy in order to understand possible challenges for Cloud Service Providers (CSPs) in d…
Platform virtualization helps solving major grid computing challenges: share resource with flexible, user-controlled and custom execution environments and in the meanwhile, isolate failures and malicious code. Grid resource management tools will evolve to embrace support for virtual resource. We present two open source projects that transparently supply virtual execution environments. Tycoon has been developed at HP Labs to optimise resource usage in creating an economy where users bid to access virtual machines and compete for CPU cycles. SmartDomains provides a peer-to-peer layer that automa…
The biggest overhead for the instantiation of a virtual machine in a cloud infrastructure is the time spent in transferring the image of the virtual machine into the physical node that executes it. This overhead becomes larger for requests composed of several virtual machines to be started concurrently, and the illusion of flexibility and elasticity usually associated with the cloud computing model may vanish. This poses a problem for both the resource providers and the software developers, since tackling those overheads is not a trivial issue. In this work we implement and evaluate several im…
Cloud computing datacenters provide thousands to millions of virtual machines (VMs) on-demand in highly dynamic environments, requiring quick placement of requested VMs into available physical machines (PMs). Due to the randomness of customer requests, the Virtual Machine Placement (VMP) should be formulated as an online optimization problem. The first part of this work analyzes alternatives to solve the formulated problem, an experimental comparison of five different online deterministic heuristics against an offline memetic algorithm with migration of VMs was performed, considering several e…
In Mobile Cloud Computing (MCC), Virtual Machine (VM) migration based process offloading is a dominant approach to enhance Smart Mobile Devices (SMDs). A challenging aspect of VM deployment is the additional computing resources usage in the deployment and management of VM which obliges computing resources for VM creation and configuration. The management of VM comprises computing resources exploitation in the monitoring of VM in entire lifecycle and physical resources management for VM on SMDs. Therefore, VM migration based application offloading requires additional computing resource. Consequ…
Stream processing applications extract value from raw data through Directed Acyclic Graphs of data analysis tasks. Shared-nothing (SN) parallelism is the de-facto standard to scale stream processing applications. Given an application, SN parallelism instantiates several copies of each analysis task, making each instance responsible for a dedicated portion of the overall analysis, and relies on dedicated queues to exchange data among connected instances. On the one hand, SN parallelism can scale the execution of applications both up and out since threads can run task instances within and across…
The use of cloud technology can offer significant savings for the deployment of services, provided that the service is able to make efficient use of the available virtual resources to meet service-level requirements. To avoid software designs that scale poorly, it is important to make deployment decisions for the service at design time, early in the development of the service itself. ABS offers a formal, model-based approach which integrates the design of services with the modeling of deployment decisions. In this paper, we illustrate the main concepts of this approach by modeling a scalable p…
IoT deployments have been growing manifold, encompassing sensors, networks, edge, fog and cloud resources. Despite the intense interest from researchers and practitioners, most do not have access to large-scale IoT testbeds for validation. Simulation environments that allow analytical modeling are a poor substitute for evaluating software platforms or application workloads in realistic computing environments. Here, we propose VIoLET, a virtual environment for defining and launching large-scale IoT deployments within cloud VMs. It offers a declarative model to specify container-based compute re…
Container technologies, like Docker, are becoming increasingly popular. Containers provide exceptional developer experience because containers offer lightweight isolation and ease of software distribution. Containers are also widely used in production environments, where a different set of challenges arise such as security, networking, service discovery and load balancing. Container cluster management tools, such as Kubernetes, attempt to solve these problems by introducing a new control layer with the container as the unit of deployment. However, adding a new control layer is an extra configu…
Temple University（天普大学）；King Fahd University of Petroleum & Minerals（法赫德国王石油与矿业大学）
In many applications, robots autonomous deployment is preferable and sometimes it is the only affordable solution. To address this issue, virtual force (VF) is one of the prominent approaches to performing multirobot deployment autonomously. However, most of the existing VF-based approaches consider only a uniform deployment to maximize the covered area while ignoring the criticality of specific locations during the deployment process. To overcome these limitations, we present a framework for autonomously deploy robots or vehicles using virtual force. The framework is composed of two stages. I…
Virtualization technology and cloud computing have brought a paradigm shift in the way we utilize, deploy and manage computer resources. They allow fast deployment of multiple operating system as containers on physical ma- chines which can be either discarded after use or check- pointed for later re-deployment. At European Organization for Nuclear Research (CERN), we have been using virtualization technology to quickly setup virtual machines for our developers with preconfigured software to enable them to quickly test/deploy a new version of a software patch for a given application. This paper…
The IP Multimedia Subsystem (IMS) defined by the 3GPP has been mainly developed and deployed by telephony vendors on vendor-specific hardware. Recent advances in Network Function Virtualisation (NFV) technology paved the way for virtualized hardware and telephony function elasticity. As such, Telecom vendors have started to embrace the cloud as a deployment platform, usually selecting a privileged virtualization platform. Operators would like to deploy telecom functionality on their already existing IT cloud platforms. Achieving such flexibility would require the telecom vendors to adopt a sof…
In the cloud computing environment, cloud virtual machine (VM) will be more and more the number of virtual machine security and management faced giant Challenge. In order to address security issues cloud computing virtualization environment, this paper presents a virtual machine based on efficient and dynamic deployment VM security management model state migration and scheduling, study of which virtual machine security architecture, based on AHP (Analytic Hierarchy Process) virtual machine deployment and scheduling method, based on CUSUM (Cumulative Sum) DDoS attack detection algorithm, and th…
Elasticity is one important feature in modern cloud computing systems and can result in computation failure or significantly increase computing time. Such elasticity means that virtual machines over the cloud can be preempted under a short notice (e.g., hours or minutes) if a high-priority job appears; on the other hand, new virtual machines may become available over time to compensate the computing resources. Coded Storage Elastic Computing (CSEC) introduced by Yang et al. in 2018 is an effective and efficient approach to overcome the elasticity and it costs relatively less storage and comput…
Transilvania University of Brasov（布拉索夫特兰西瓦尼亚大学）；Elektrobit Automotive（易特驰汽车）；University of Southampton（南安普顿大学）
Self-driving cars and autonomous vehicles are revolutionizing the automotive sector, shaping the future of mobility altogether. Although the integration of novel technologies such as Artificial Intelligence (AI) and Cloud/Edge computing provides golden opportunities to improve autonomous driving applications, there is the need to modernize accordingly the whole prototyping and deployment cycle of AI components. This paper proposes a novel framework for developing so-called AI Inference Engines for autonomous driving applications based on deep learning modules, where training tasks are deployed…
This paper addresses the impact of Virtual Memory Streaming (VMS) technique in provisioning virtual machines (VMs) in cloud environment. VMS is a scaling virtualization technology that allows different virtual machines rapid scale, high performance, and increase hardware utilization. Traditional hypervisors do not support true no-downtime live migration, and its lack of memory oversubscription can hurt the economics of a private cloud deployment by limiting the number of VMs on each host. VMS brings together several advanced hypervisor memory management techniques including granular page shari…
State-of-the-art deep learning systems such as TensorFlow and PyTorch tightly couple the model with the underlying hardware. This coupling requires the user to modify application logic in order to run the same job across a different set of resources, thereby limiting the choice of hardware for a given workload and potentially forcing the user to forgo more efficient hardware configurations. We propose VirtualFlow, a system leveraging a novel abstraction called virtual node processing to decouple the model from the hardware. In each step of training or inference, the batch of input data is spli…
Peer-to-peer (P2P) networks have mostly focused on task oriented networking, where networks are constructed for single applications, i.e. file-sharing, DNS caching, etc. In this work, we introduce IPOP, a system for creating virtual IP networks on top of a P2P overlay. IPOP enables seamless access to Grid resources spanning multiple domains by aggregating them into a virtual IP network that is completely isolated from the physical network. The virtual IP network provided by IPOP supports deployment of existing IP-based protocols over a robust, self-configuring P2P overlay. We present implement…
WISDOM is an international initiative to enable a virtual screening pipeline on a grid infrastructure. Its first attempt was to deploy large scale in silico docking on a public grid infrastructure. Protein-ligand docking is about computing the binding energy of a protein target to a library of potential drugs using a scoring algorithm. Previous deployments were either limited to one cluster, to grids of clusters in the tightly protected environment of a pharmaceutical laboratory or to pervasive grids. The first large scale docking experiment ran on the EGEE grid production service from 11 July…
