Cloud native computing paradigm allows microservice-based applications to take advantage of cloud infrastructure in a scalable, reusable, and interoperable way. However, in a cloud native system, the vast number of configuration parameters and highly granular resource allocation policies can significantly impact the performance and deployment cost. For understanding and analyzing these implications in an easy, quick, and cost-effective way, we present PerfSim, a discrete-event simulator for approximating and predicting the performance of cloud native service chains in user-defined scenarios. To this end, we proposed a systematic approach for modeling the performance of microservices endpoint functions by collecting and analyzing their performance and network traces. With a combination of the extracted models and user-defined scenarios, PerfSim can then simulate the performance behavior of all services over a given period and provide an approximation for system KPIs, such as requests' average response time. Using the processing power of a single laptop, we evaluated both simulation accuracy and speed of PerfSim in 104 prevalent scenarios and compared the simulation results with the identical deployment in a real Kubernetes cluster. We achieved ~81-99% simulation accuracy in approximating the average response time of incoming requests and ~16-1200 times speed-up factor for the simulation.
CloudNativeSim: a toolkit for modeling and simulation of cloud-native applications
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院);University of Chinese Academy of Sciences(中国科学院大学);Southern University of Science and Technology(南方科技大学);State Key Lab of IOTSC, Department of Computer Science, University of Macau(澳门大学计算机科学与信息技术学院物联网与智慧城市重点实验室)
A Cloud-Native Architecture for Human-in-Control LLM-Assisted OpenSearch in Investigative Settings
AI Research Unit of the State Police of Schleswig-Holstein(石勒苏益格-荷尔斯泰因州警察AI研究部);Faculty of Computer Science and Electrical Engineering Kiel University of Applied Sciences(基尔应用科技大学计算机科学与电气工程学院)
ElegantRL-Podracer: Scalable and Elastic Library for Cloud-Native Deep Reinforcement Learning
Columbia University(哥伦比亚大学);University of California, Berkeley(加州大学伯克利分校);Shenzhen Inst. of Advanced Tech.(深圳先进技术研究院);Northwestern University(西北大学);Bell labs(贝尔实验室);Amazon(亚马逊);IDEA Research(IDEA研究院)
Cloud-native and Distributed Systems for Efficient and Scalable Large Language Models -- A Research Agenda
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院);University of Hyderabad(海德拉巴大学);Western Sydney University(西悉尼大学);Newcastle University(纽卡斯尔大学);Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of Electronic Science and Technology of China(电子科技大学);University of Macau(澳门大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Carleton University(卡尔顿大学);University of Ljubljana(卢布尔雅那大学);University of Melbourne(墨尔本大学)