Microsoft Azure is dedicated to guarantee high quality of service to its customers, in particular, during periods of high customer activity, while controlling cost. We employ a Data Science (DS) driven solution to predict user load and leverage these predictions to optimize resource allocation. To this end, we built the Seagull infrastructure that processes per-server telemetry, validates the data, trains and deploys ML models. The models are used to predict customer load per server (24h into the future), and optimize service operations. Seagull continually re-evaluates accuracy of predictions, fallback to previously known good models and triggers alerts as appropriate. We deployed this infrastructure in production for PostgreSQL and MySQL servers across all Azure regions, and applied it to the problem of scheduling server backups during low-load time. This minimizes interference with user-induced load and improves customer experience.
A Home Area Network (HAN) is considered to be a significant component of Advanced Metering Infrastructure (AMI) and has been studied well in many works. It binds all the electrical components installed in a defined premise together for their close monitoring and management. However, HAN has been realized so far mostly as a centralized system. Therefore, like any other centralized system, the traditional realization of HAN also suffers from various well-known problems, such as single-point-of-failure, susceptibility to attacks, requirement of specialized infrastructure, inflexibility to easy ex…
BanaServe: Unified KV Cache and Dynamic Module Migration for Balancing Disaggregated LLM Serving in AI Infrastructure
Southern University of Science and Technology（南方科技大学）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；University of Chinese Academy of Sciences（中国科学院大学）；State Key Lab of IOTSC, Faculty of Science and Technology, University of Macau（澳门科技大学物联网与感知系统国家重点实验室、理学院）；AIOS Team, Alibaba Group Inc（阿里巴巴集团 AIOS 团队）
Serverless computing is an emerging service model in distributed computing systems. The term captures cloud-based event-driven distributed application design and stems from its completely resource-transparent deployment model, i.e. serverless. This work thesisizes that adaptive event dispatching can improve current serverless platform resource efficiency by considering locality and dependencies. These design contemplations have also been formulated by Hendrickson et al., which identifies the requirement that "Serverless load balancers must make low-latency decisions while considering session, …
In this article we look at the potential of cloud containers and we provide some guidelines for companies and organisations that are starting to look at how to migrate their legacy infrastructure to something modern, reliable and scalable. We propose an architecture that has an excellent relationship between the cost of implementation and the benefits it can bring, based on the "Pilot Light" topology. The services are reconfigured inside small docker containers and the workload is balanced using load balancers that allow horizontal autoscaling techniques to be exploited in the future. By gener…
Compute infrastructure hosted by a cloud provider allows an application to scale without limit. The application developer no longer needs to worry about the up-front investment in a server farm provisioned for a worst-case load scenario. However, managing cloud deployments requires a sophisticated framework that can autoscale the infrastructure and guarantee the up-time of running container images. This paper surveys existing research addressing the management and orchestration of cloud deployments as well as the modelling framework to abstract away the low-level details of the host infrastruc…
Every day the number of traffic cameras in cities rapidly increase and huge amount of video data are generated. Parallel processing infrastruture, such as Hadoop, and programming models, such as MapReduce, are being used to promptly process that amount of data. The common approach for video processing by using Hadoop MapReduce is to process an entire video on only one node, however, in order to avoid parallelization problems, such as load imbalance, we propose to process videos by splitting it into equal parts and processing each resulting chunk on a different node. We used some machine learni…
As Smart Meters are collecting and transmitting household energy consumption data to Retail Energy Providers (REP), the main challenge is to ensure the effective use of fine-grained consumer data while ensuring data privacy. In this manuscript, we tackle this challenge for energy load consumption forecasting in regards to REPs which is essential to energy demand management, load switching and infrastructure development. Specifically, we note that existing energy load forecasting is centralized, which are not scalable and most importantly, vulnerable to data privacy threats. Besides, REPs are i…
Financial markets are event- and data-driven to an extremely high degree. For making decisions and triggering actions stakeholders require notifications about significant events and reliable background information that meet their individual requirements in terms of timeliness, accuracy, and completeness. As one of Europe's leading providers of financial data and regulatory solutions vwd processes an average of 18 billion event notifications from 500+ data sources for 30 million symbols per day. Our large-scale distributed event-based systems handle daily peak rates of 1+ million event notifica…
Service function chaining (SFC) is promising to implement flexible and scalable virtual network infrastructure for the Internet of Things (IoT). Edge computing is envisioned to be an effective solution to process huge amount of IoT application data. In order to uniformly provide services to IoT applications among the distributed edge computing nodes (ECNs), we present a unified SFC orchestration framework based on the coordination of SDN and NFV, which provides a synergic edge cloud platform by exploiting the connectivity of FiWi access networks. In addition, we study the VNF deployment proble…
