The environmental impact of Large Language Models (LLMs) on data centers hosting these models is becoming a significant concern. While many efforts have focused on reducing the substantial training overhead of LLMs, carbon and water consumption during the inference phase can often surpass the costs associated with their training. The cooling systems of data centers are crucial in this context, but they are frequently modeled with a location-independent efficiency term. However, their energy efficiency is highly influenced by ambient temperature, which can vary significantly across different geographical locations. Leveraging this temperature diversity can help reduce total cooling energy costs and improve the performance of edge data centers. To address these critical sustainability issues related to LLMs, this study proposes a temperature-aware approach that co-optimizes LLM energy costs, carbon emissions, time-to-first token, and water consumption. The approach employs a distributed optimization algorithm based on an alternating direction method of multipliers, aimed at enhancing the sustainability of LLM hosting across geo-distributed edge data centers in Australia. Our method demonstrates reductions in cooling energy consumption and improves overall cost efficiency for geo-distributed cloud environments.
Aging-Aware Online Distributed Scheduling for Lifecycle Carbon Reduction in Geo-Distributed Data Centers
Guangzhou University(广州大学);The Chinese University of Hong Kong (Shenzhen)(香港中文大学(深圳));Shenzhen Research Institute of Big Data(深圳大数据研究院);Rausser College of Natural Resources, UC Berkeley(加州大学伯克利分校罗斯尔自然资源学院)
Machine Learning and CPU (Central Processing Unit) Scheduling Co-Optimization over a Network of Computing Centers
Mechatronics Group, Faculty of Mechanical Engineering, Semnan University, Semnan, Iran(塞姆恩大学机械电子组,机械工程学院);Center for International Scientific Studies(国际科学研究中心);Collaborations, Tehran, Iran(合作,德黑兰伊朗);Institute of Computational Mathematics and Information Technologies(计算数学与信息科技研究所);Kazan Federal University(卡赞联邦大学);Computer Engineering Department, Sharif University of Technology(技术大学计算机工程系)
Energy Management for Renewable-Colocated Artificial Intelligence Data Centers
School of Electrical and Computer Engineering, Cornell University(电气与计算机工程系,康奈尔大学);Dyson School of Applied Economics and Management, Cornell University(应用经济学与管理学院,康奈尔大学)