基于大语言模型的意图驱动的隐私意识云-边连续体协同
LLM-Driven Intent-Based Privacy-Aware Orchestration Across the Cloud-Edge Continuum
AI总结：
本文提出了一种动态管道重新配置方法，用于在云-边连续体上实现意图驱动的隐私意识协同，通过减少服务停机时间和性能损失来优化LLM推理效率。
摘要：
随着大语言模型（LLMs）的快速发展，如何在有限的GPU资源下高效地服务LLM推理已成为一个关键挑战。最近，越来越多的研究探索了将无服务器计算范式应用于LLM服务，以最大限度地利用资源。然而，LLM推理工作负载高度多样，而现代GPU集群本质上是异构的，这使得必须动态调整部署配置以更好地适应无服务器环境的弹性和动态性。同时，由于LLM推理的有状态性质和模型参数的庞大规模，实现这种在线重新配置尤其具有挑战性。在本文中，我们提出了一种动态管道重新配置方法，该方法可以在最小化服务停机时间和性能下降的同时实现管道配置的在线调整。我们的方法允许系统根据变化的工作负载选择最佳的管道配置。在异构GPU平台（包括NVIDIA A100和L40s）上的实验结果表明，我们的迁移机制导致的服务停机时间少于50毫秒，同时在时间到第一个标记（TTFT）和每个输出标记的时间（TPOT）上引入了不到10%的开销。
英文摘要：
With the rapid advancement of large language models (LLMs), efficiently serving LLM inference under limited GPU resources has become a critical challenge. Recently, an increasing number of studies have explored applying serverless computing paradigms to LLM serving in order to maximize resource utilization. However, LLM inference workloads are highly diverse, and modern GPU clusters are inherently heterogeneous, making it necessary to dynamically adjust deployment configurations online to better adapt to the elastic and dynamic nature of serverless environments. At the same time, enabling such online reconfiguration is particularly challenging due to the stateful nature of LLM inference and the massive size of model parameters. In this paper, we propose a dynamic pipeline reconfiguration approach that enables online adjustment of pipeline configurations while minimizing service downtime and performance degradation. Our method allows the system to select the optimal pipeline configuration in response to changing workloads. Experimental results on heterogeneous GPU platforms, including NVIDIA A100 and L40s, demonstrate that our migration mechanism incurs less than 50 ms of service downtime, while introducing under 10% overhead on both time-to-first-token (TTFT) and time-per-output-token (TPOT).
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
