RAGPulse:一个用于优化RAG服务系统的开源RAG工作负载轨迹
RAGPulse: An Open-Source RAG Workload Trace to Optimize RAG Serving Systems
AI总结:
针对现有通用LLM推理轨迹无法捕捉RAG系统特有工作负载特征的问题,本文推出采集自真实校园问答系统的开源RAG工作负载轨迹数据集RAGPulse,为RAG服务优化提供高保真研究基础。
摘要:
检索增强生成(Retrieval-Augmented Generation, RAG)是构建可靠、知识密集型大语言模型(Large Language Model, LLM)应用的关键范式。然而,RAG系统的多阶段流水线(检索、生成)和独特工作负载特征(如知识依赖性)给服务性能优化带来了巨大挑战。现有的通用LLM推理轨迹无法捕捉这些RAG特有的动态特性,造成了学术研究与真实部署之间显著的性能差距。为弥补这一差距,本文推出了开源RAG工作负载轨迹数据集RAGPulse。该数据集采集自2024年4月起投入使用、服务超4万名师生的全校级问答系统。本文详细介绍了RAGPulse的系统架构、基于哈希的隐私保护数据格式,并开展了深入的统计分析。分析发现,真实世界RAG工作负载呈现出显著的时间局部性和高度倾斜的热门文档访问模式。RAGPulse为研究人员开发、验证内容感知批处理、检索缓存等新型RAG系统优化策略提供了高保真基础,最终可提升RAG服务的效率与可靠性。代码可在https://github.com/flashserve/RAGPulse获取。
英文摘要:
Retrieval-Augmented Generation (RAG) is a critical paradigm for building reliable, knowledge-intensive Large Language Model (LLM) applications. However, the multi-stage pipeline (retrieve, generate) and unique workload characteristics (e.g., knowledge dependency) of RAG systems pose significant challenges for serving performance optimization. Existing generic LLM inference traces fail to capture these RAG-specific dynamics, creating a significant performance gap between academic research and real-world deployment. To bridge this gap, this paper introduces RAGPulse, an open-source RAG workload trace dataset. This dataset was collected from an university-wide Q&A system serving that has served more than 40,000 students and faculties since April 2024. We detail RAGPulse's system architecture, its privacy-preserving hash-based data format, and provide an in-depth statistical analysis. Our analysis reveals that real-world RAG workloads exhibit significant temporal locality and a highly skewed hot document access pattern. RAGPulse provides a high-fidelity foundation for researchers to develop and validate novel optimization strategies for RAG systems, such as content-aware batching and retrieval caching, ultimately enhancing the efficiency and reliability of RAG services. The code is available at https://github.com/flashserve/RAGPulse.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。