arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2503.08415cs.DC
收藏

TokenSim:面向大语言模型推理系统的软硬件探索支撑框架

TokenSim: Enabling Hardware and Software Exploration for Large Language Model Inference Systems

Feiyang Wu, Zhuohang Bian, Guoyang Duan, Tianle Xu, Junchi Wu, Teng Ma, Yongqiang Yao, Ruihao Gong, Youwei Zhuo

AI总结:

针对大语言模型推理系统对高效可扩展服务的需求,本文提出软硬件探索系统TokenSim,支持调度、内存管理等可扩展优化,在真实数据集下误差率低于1%,可支撑LLM服务系统的性能与优化探索。

摘要:

大语言模型(LLM)服务需求的持续增长,迫切需要推动LLM推理系统的优化与性能剖析技术取得重大进展。随着这类模型成为各类应用的核心组件,高效可扩展的服务解决方案需求呈指数级增长。本文提出TokenSim——一款专门针对LLM推理设计的综合性软硬件探索系统。TokenSim的核心特点是支持可扩展的系统优化,涵盖调度与内存管理模块。我们采用真实数据集在实际运行的系统上验证结果,实现了低于1%的误差率。此外,TokenSim还支持对LLM服务系统的性能与优化开展各类深度探索研究。

英文摘要:

The increasing demand for large language model (LLM) serving has necessitated significant advancements in the optimization and profiling of LLM inference systems. As these models become integral to a wide range of applications, the need for efficient and scalable serving solutions has grown exponentially. This work introduces TokenSim, a comprehensive hardware and software exploration system designed specifically for LLM inference. TokenSim is characterized by its support for extensible system optimizations including scheduling and memory management. We validate the results with systems running with realworld datasets, achieving an error rate of less than 1%. Furthermore, TokenSim facilitates various insightful explorations into the performance and optimization of LLM serving systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑