LLM 服务的 CAP 原理:长上下文大语言模型服务综述
The CAP Principle for LLM Serving: A Survey of Long-Context Large Language Model Serving
AI总结:
本综述提出 LLM 服务的 CAP 原理,指出上下文长度、准确性与性能三者最多同时优化两项,并分类梳理了长上下文服务相关研究。
摘要:
我们综述了大语言模型(LLM)服务领域,以理解成本效率与准确性之间错综复杂的动态关系,而这种关系因大规模部署模型时对更长上下文理解需求的日益增长而被放大。我们的研究发现,该领域的工作沿三个不同但相互冲突的目标进行优化:提升服务上下文长度(C)、提升服务准确性(A)以及提升服务性能(P)。受数据库中 CAP 定理的启发,我们提出了 LLM 服务的 CAP 原理,该原理表明任何优化最多只能同时改善这三个目标中的两个。本综述将现有工作归类于这一框架内。我们发现,用户感知测量指标的定义和连续性对于确定某个目标是否已达成至关重要,这类似于现实世界中先前的 CAP 数据库。我们将 LLM 服务的 CAP 原理视为一个指导性原则,而非正式定理,以告知设计者服务模型中固有且动态的权衡。由于服务准确性和性能已被广泛研究,本综述聚焦于那些扩展服务上下文长度并应对由此产生的挑战的工作。
英文摘要:
We survey the large language model (LLM) serving area to understand the intricate dynamics between cost-efficiency and accuracy, which is magnified by the growing need for longer contextual understanding when deploying models at a massive scale. Our findings reveal that works in this space optimize along three distinct but conflicting goals: improving serving context length (C), improving serving accuracy (A), and improving serving performance (P). Drawing inspiration from the CAP theorem in databases, we propose a CAP principle for LLM serving, which suggests that any optimization can improve at most two of these three goals simultaneously. Our survey categorizes existing works within this framework. We find the definition and continuity of user-perceived measurement metrics are crucial in determining whether a goal has been met, akin to prior CAP databases in the wild. We recognize the CAP principle for LLM serving as a guiding principle, rather than a formal theorem, to inform designers of the inherent and dynamic trade-offs in serving models. As serving accuracy and performance have been extensively studied, this survey focuses on works that extend serving context length and address the resulting challenges.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。