RAGOps：检索增强生成流水线的运营与管理
RAGOps: Operating and Managing Retrieval-Augmented Generation Pipelines
AI总结：
本文提出RAGOps框架，刻画RAG应用架构与生命周期，定义各阶段设计考量和质量权衡，并给出研究挑战与用例。
摘要：
近期研究表明，企业环境中60%基于LLM的复合系统采用了某种形式的检索增强生成（RAG），通过从外部数据源检索相关信息来提升LLM（或其他生成式AI）输出的相关性和准确性。LLMOps涵盖在生产环境中管理LLM复合系统生命周期与运营的实践和技术，支持通过持续运营和反馈评估来增强LLM系统。RAGOps在LLMOps基础上扩展，重点关注数据管理，以应对外部数据源的持续变化。这需要自动化方法来评估和测试数据操作，提升检索相关性和生成质量。本文中，我们（1）基于描述软件架构的4+1模型视图刻画RAG应用的通用架构；（2）概述RAG系统的生命周期，该生命周期整合了LLM和数据两者的管理生命周期；（3）定义RAGOps在RAG生命周期不同阶段的关键设计考量及质量权衡分析；（4）突出围绕RAGOps的总体研究挑战；（5）呈现两个RAG应用用例及相应的RAGOps考量。
英文摘要：
Recent studies show that 60% of LLM-based compound systems in enterprise environments leverage some form of retrieval-augmented generation (RAG), which enhances the relevance and accuracy of LLM (or other genAI) outputs by retrieving relevant information from external data sources. LLMOps involves the practices and techniques for managing the lifecycle and operations of LLM compound systems in production environments. It supports enhancing LLM systems through continuous operations and feedback evaluation. RAGOps extends LLMOps by incorporating a strong focus on data management to address the continuous changes in external data sources. This necessitates automated methods for evaluating and testing data operations, enhancing retrieval relevance and generation quality. In this paper, we (1) characterize the generic architecture of RAG applications based on the 4+1 model view for describing software architectures, (2) outline the lifecycle of RAG systems, which integrates the management lifecycles of both the LLM and the data, (3) define the key design considerations of RAGOps across different stages of the RAG lifecycle and quality trade-off analyses, (4) highlight the overarching research challenges around RAGOps, and (5) present two use cases of RAG applications and the corresponding RAGOps considerations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
