Ensuring data correctness over partitioned distributed database systems is a classical problem. Classical solutions proposed to solve this problem are mainly adopting locking or blocking techniques. These techniques are not suitable for cloud environments as they produce terrible response times; due to the long latency and faultiness of wide area network connections among cloud datacenters. One way to improve performance is to restrict access of users-bases to specific datacenters and avoid data sharing between datacenters. However, conflicts might appear when data is replicated between datacenters; nevertheless change propagation timeliness is not guaranteed. Such problems created data uncertainty on cloud environments. Managing data uncertainty is one of the main obstacles for supporting global distributed transactions on the clouds. To overcome this problem, this paper proposes an quota-based approach for managing data uncertainty on the clouds that guarantees global data correctness without global locking or blocking. To decouple service developers from the hassles of managing data uncertainty, we propose to use a new platform service (i.e. Data Consistency as a Service (DCaaS)) to encapsulate the proposed approach. DCaaS service also ensures SaaS services cloud portability, as it works as a cloud adapter between SaaS service instances. Experiments show that proposed approach realized by the DCaaS service provides much better response time when compared with classical locking and blocking techniques.
Interactions between cloud services result in service dependencies. Evaluating and managing the cascading impacts caused by service dependencies is critical to the reliability of cloud systems. This paper summarizes the dependency types in cloud systems and demonstrates the design of the Dependency Management System (DMS), a platform for managing the service dependencies in the production cloud system. DMS features full-lifecycle support for service reliability (i.e., initial service deployment, service upgrade, proactive architectural optimization, and reactive failure mitigation) and refined…
Multi-cloud computing is a promising paradigm to support very large scale world wide distributed applications. Multi-cloud computing is the usage of multiple, independent cloud environments, which assumed no priori agreement between cloud providers or third party. However, multi-cloud computing has to face several key challenges such as portability, provisioning, elasticity, and high availability. Developers will not only have to deploy applications to a specific cloud, but will also have to consider application portability from one cloud to another, and to deploy distributed applications span…
Online data-intensive services parallelize query execution across distributed software components. Interactive response time is a priority, so online query executions return answers without waiting for slow running components to finish. However, data from these slow components could lead to better answers. We propose Ubora, an approach to measure the effect of slow running components on the quality of answers. Ubora randomly samples online queries and executes them twice. The first execution elides data from slow components and provides fast online answers; the second execution waits for all c…
Computational resource provisioning that is closer to a user is becoming increasingly important, with a rise in the number of devices making continuous service requests and with the significant recent take up of latency-sensitive applications, such as streaming and real-time data processing. Fog computing provides a solution to such types of applications by bridging the gap between the user and public/private cloud infrastructure via the inclusion of a "fog" layer. Such approach is capable of reducing the overall processing latency, but the issues of redundancy, cost-effectiveness in utilizing…
The Grid technology is evolving into a global, service-orientated architecture, a universal platform for delivering future high demand computational services. Strong adoption of the Grid and the utility computing concept is leading to an increasing number of Grid installations running a wide range of applications of different size and complexity. In this paper we address the problem of elivering deadline/economy based scheduling in a heterogeneous application environment using statistical properties of job historical executions and its associated meta-data. This approach is motivated by a stud…
Mobile edge computing (MEC) and next-generation mobile networks are set to disrupt the way intelligent and autonomous systems are interconnected. This will have an effect on a wide range of domains, from the Internet of Things to autonomous mobile robots. The integration of such a variety of MEC services in a inherently distributed architecture requires a robust system for managing hardware resources, balancing the network load and securing the distributed applications. Blockchain technology has emerged a solution for managing MEC services, with consensus protocols and data integrity checks th…
One of the fundamental concepts in Grid computing is the creation of Virtual Organizations (VO's): a set of resource consumers and providers that join forces to solve a common problem. Typical examples of Virtual Organizations include collaborations formed around the Large Hadron Collider (LHC) experiments. To date, Grid computing has been applied on a relatively small scale, linking dozens of users to a dozen resources, and management of these VO's was a largely manual operation. With the advance of large collaboration, linking more than 10000 users with a 1000 sites in 150 counties, a compre…
Rapid adoption of Cloud computing for hosting services and its success is primarily attributed to its attractive features such as elasticity, availability and pay-as-you-go pricing model. However, the huge amount of energy consumed by cloud data centers makes it to be one of the fastest growing sources of carbon emissions. Approaches for improving the energy efficiency include enhancing the resource utilization to reduce resource wastage and applying the renewable energy as the energy supply. This work aims to reduce the carbon footprint of the data centers by reducing the usage of brown energ…
Modern Cyber-physical Systems (CPS) include applications like smart traffic, smart agriculture, smart power grid, etc. Commonly, these systems are distributed and composed of end-user applications and microservices that typically run in the cloud. The connection with the physical world, which is inherent to CPS, brings the need to operate and respond in real-time. As the cloud becomes part of the computation loop, the real-time requirements have to be also reflected by the cloud. In this paper, we present an approach that provides soft real-time guarantees on the response time of services runn…
This paper reports on experience with using semantically-enabled network resource models to construct an operational multi-domain networked infrastructure-as-a-service (NIaaS) testbed called ExoGENI, recently funded through NSF's GENI project. A defining property of NIaaS is the deep integration of network provisioning functions alongside the more common storage and computation provisioning functions. Resource provider topologies and user requests can be described using network resource models with common base classes for fundamental cyber-resources (links, nodes, interfaces) specialized via v…
The architectural style of microservices has been gaining popularity in recent years. In this architectural style, small and loosely coupled modules are deployed and scaled independently to compose cloud-native applications. Carrier-grade service providers are migrating their legacy applications to a microservice based architecture running on Kubernetes which is an open source platform for orchestrating containerized microservice based applications. However, in this migration, service availability remains a concern. Service availability is measured as the percentage of time the service is prov…
Current IoT services include industry-oriented services, which often require objects to run more than one task. However, the exponential growth of objects in IoT poses the challenge of distributing and managing task allocation among objects. One of the main goals of task allocation is to improve the quality of information and maximize the tasks to be performed. Although there are approaches that optimize and manage the dynamics of nodes, not all consider the quality of information and the distributed allocation over the cluster service. This paper proposes the mechanism CONTASKI for task alloc…
Managing cloud services is a fundamental challenge in todays virtualized environments. These challenges equally face both providers and consumers of cloud services. The issue becomes even more challenging in virtualized environments that support mobile clouds. Cloud computing platforms such as Amazon EC2 provide customers with flexible, on demand resources at low cost. However, they fail to provide seamless infrastructure management and monitoring capabilities that many customers may need. For instance, Amazon EC2 doesn't fully support cloud services automated discovery and it requires a priva…
A fundamental ambition of grid and distributed systems is to be capable of sustaining evolution and allowing for adaptability ((F. Losavio et al., 2002), (S. Radhakrishnan, 2005)). Furthermore, as the complexity and sophistication of theses structures increases, so does the need for adaptability of each component. One of the primary benefits of service oriented architecture (SOA) is the ability to compose applications, processes or more complex services from other services which increases the capacity for adaptation. This document proposes a novel infrastructure composition model that aims at …
The scope of this project's work focuses on the research and prototyping of the extension of the Distributed MARF such that its services can be managed through the most popular management protocol familiarly, SNMP. The rationale behind SNMP vs. MARF's proprietary management protocols, is that can be integrated with the use of common network service and device management, so the administrators can manage MARF nodes via a already familiar protocol, as well as monitor their performance, gather statistics, set desired configuration, etc. perhaps using the same management tools they've been using f…
This chapter presents: (i) a layered peer-to-peer Cloud provisioning architecture; (ii) a summary of the current state-of-the-art in Cloud provisioning with particular emphasis on service discovery and load-balancing; (iii) a classification of the existing peer-to-peer network management model with focus on extending the DHTs for indexing and managing complex provisioning information; and (iv) the design and implementation of novel, extensible software fabric (Cloud peer) that combines public/private clouds, overlay networking and structured peer-to-peer indexing techniques for supporting scal…
funcX is a distributed function as a service (FaaS) platform that enables flexible, scalable, and high performance remote function execution. Unlike centralized FaaS systems, funcX decouples the cloud-hosted management functionality from the edge-hosted execution functionality. funcX's endpoint software can be deployed, by users or administrators, on arbitrary laptops, clouds, clusters, and supercomputers, in effect turning them into function serving systems. funcX's cloud-hosted service provides a single location for registering, sharing, and managing both functions and endpoints. It allows f…
FaaS (Function as a Service) allows developers to upload and execute code in the cloud without managing servers. FaaS offerings from leading public cloud providers are based on system microVM or application container technologies such as Firecracker or Docker. In this paper, we demonstrate that lightweight high-level runtimes, such as WebAssembly, could offer performance and scaling advantages over existing solutions, and could enable finely-grained pay-as-you-use business models. We compared widely used performance benchmarks between Docker native and WebAssembly implementations of the same a…
The powerful paradigm of Fog computing is currently receiving major interest, as it provides the possibility to integrate virtualized servers into networks and brings cloud service closer to end devices. To support this distributed intelligent platform, Software-Defined Network (SDN) has emerged as a viable network technology in the Fog computing environment. However, uncertainties related to task demands and the different computing capacities of Fog nodes, inquire an effective load balancing algorithm. In this paper, the load balancing problem has been addressed under the constraint of achiev…
Many scientific high-throughput applications can benefit from the elastic nature of Cloud resources, especially when there is a need to reduce time to completion. Cost considerations are usually a major issue in such endeavors, with networking often a major component; for data-intensive applications, egress networking costs can exceed the compute costs. Dedicated network links provide a way to lower the networking costs, but they do add complexity. In this paper we provide a description of a 100 fp32 PFLOPS Cloud burst in support of IceCube production compute, that used Internet2 Cloud Connect…
Energy consumption of Cloud data centers has been a major concern of many researchers, and one of the reasons for huge energy consumption of Clouds lies in the inefficient utilization of computing resources. Besides energy consumption, another challenge of data centers is the unexpected loads, which leads to the overloads and performance degradation. Compared with VM consolidation and Dynamic Voltage Frequency Scaling that cannot function well when the whole data center is overloaded, brownout has shown to be a promising technique to handle both overloads and energy consumption through dynamic…
Cloud computing has emerged as an important paradigm for managing and delivering services efficiently over the Internet. Convergence of cloud computing with technologies such as wireless sensor networking and mobile computing offers new applications of cloud services but this requires management of Quality of Service (QoS) parameters to efficiently monitor and measure the delivered services. This paper presents a QoS-aware Cloud Based Autonomic Information System for delivering agriculture related information as a service through the use of latest Cloud technologies which manage various types …
The MammoGrid project has recently delivered its first proof-of-concept prototype using a Service-Oriented Architecture (SOA)-based Grid application to enable distributed computing spanning national borders. The underlying AliEn Grid infrastructure has been selected because of its practicality and because of its emergence as a potential open source standards-based solution for managing and coordinating distributed resources. The resultant prototype is expected to harness the use of huge amounts of medical image data to perform epidemiological studies, advanced image processing, radiographic ed…
Cloud Service Providers (CSPs) offer a wide variety of scalable, flexible, and cost-efficient services to cloud users on demand and pay-per-utilization basis. However, vast diversity in available cloud service providers leads to numerous challenges for users to determine and select the best suitable service. Also, sometimes users need to hire the required services from multiple CSPs which introduce difficulties in managing interfaces, accounts, security, supports, and Service Level Agreements (SLAs). To circumvent such problems having a Cloud Service Broker (CSB) be aware of service offerings …
Container technologies have been evolving rapidly in the cloud-native era. Kubernetes, as a production-grade container orchestration platform, has been proven to be successful at managing containerized applications in on-premises datacenters. However, Kubernetes lacks sufficient multi-tenant supports by design, meaning in cloud environments, dedicated clusters are required to serve multiple users, i.e., tenants. This limitation significantly diminishes the benefits of cloud computing, and makes it difficult to build multi-tenant software as a service (SaaS) products using Kubernetes. In this p…
Compute infrastructure hosted by a cloud provider allows an application to scale without limit. The application developer no longer needs to worry about the up-front investment in a server farm provisioned for a worst-case load scenario. However, managing cloud deployments requires a sophisticated framework that can autoscale the infrastructure and guarantee the up-time of running container images. This paper surveys existing research addressing the management and orchestration of cloud deployments as well as the modelling framework to abstract away the low-level details of the host infrastruc…
Workflow management systems (WMS) support the composition and deployment of workflow-oriented applications in distributed computing environments. They hide the complexity of managing large-scale applications, which includes the controlling data pipelining between tasks, ensuring the application's execution, and orchestrating the distributed computational resources to get a reasonable processing time. With the increasing trends of scientific workflow adoption, the demand to deploy them using a third-party service begins to increase. Workflow-as-a-service (WaaS) is a term representing the platfo…
Modern latency-critical online services often rely on composing results from a large number of server components. Hence the tail latency (e.g. the 99th percentile of response time), rather than the average, of these components determines the overall service performance. When hosted on a cloud environment, the components of a service typically co-locate with short batch jobs to increase machine utilizations, and share and contend resources such as caches and I/O bandwidths with them. The highly dynamic nature of batch jobs in terms of their workload types and input sizes causes continuously cha…
In the first part of this paper we present the first steps towards providing low-risk and economically feasible network data transfer services. We introduce three types of data transfer services and present general guidelines and algorithms for managing service prices, risks and schedules. In the second part of the paper we solve two packet scheduling cost optimization problems and present efficient algorithms for identifying maximum weight (k-level-) caterpillar subtrees in tree networks.
In this paper, we intend to answer one key question to the success of cloud computing: in cloud, do many task computing (MTC) or high throughput computing (HTC) service providers, which offer the corresponding computing service to end users, benefit from the economies of scale? Our research contributions are three-fold: first, we propose an innovative usage model, called dynamic service provision (DSP) model, for MTC or HTC service providers. In the DSP model, the resource provider provides the service of creating and managing runtime environments for MTC or HTC service providers, and consolid…
Scientific discovery is increasingly dependent on a scientist's ability to acquire, curate, integrate, analyze, and share large and diverse collections of data. While the details vary from domain to domain, these data often consist of diverse digital assets (e.g. image files, sequence data, or simulation outputs) that are organized with complex relationships and context which may evolve over the course of an investigation. In addition, discovery is often collaborative, such that sharing of the data and its organizational context is highly desirable. Common systems for managing file or asset me…
