syftr：帕累托最优生成式人工智能
syftr: Pareto-Optimal Generative AI
AI总结：
syftr框架利用贝叶斯优化和早停机制，在智能体与非智能体RAG配置中搜索帕累托最优流程，以低成本保持高准确性，显著提升生成式AI流水线效率。
摘要：
检索增强生成（RAG）流水线是将大型语言模型（LLMs）应用于专有或动态数据的核心。然而，构建有效的RAG流程十分复杂，需要在向量数据库、嵌入模型、文本分割器、检索器和合成LLMs之间进行仔细选择。随着智能体范式的兴起，这一挑战更加严峻。验证器、重写器和重排序器等模块各自具有复杂的超参数依赖关系，必须仔细调整。在性能敏感的应用中，平衡延迟、准确性和成本之间的权衡变得越来越困难。我们引入了syftr，这是一个在广泛的智能体和非智能体RAG配置空间中进行高效多目标搜索的框架。利用贝叶斯优化，syftr发现帕累托最优流程，这些流程共同优化任务准确性和成本。一种新颖的早停机制通过剪除明显次优的候选方案进一步提高了效率。在多个RAG基准测试中，syftr找到的流程平均成本约为最准确流程的九分之一，同时保留了帕累托前沿上最准确流程的大部分准确性。此外，syftr的设计和优化能力允许集成新模块，使得实现高性能生成式AI流水线更加容易和快速。
英文摘要：
Retrieval-Augmented Generation (RAG) pipelines are central to applying large language models (LLMs) to proprietary or dynamic data. However, building effective RAG flows is complex, requiring careful selection among vector databases, embedding models, text splitters, retrievers, and synthesizing LLMs. The challenge deepens with the rise of agentic paradigms. Modules like verifiers, rewriters, and rerankers-each with intricate hyperparameter dependencies have to be carefully tuned. Balancing tradeoffs between latency, accuracy, and cost becomes increasingly difficult in performance-sensitive applications. We introduce syftr, a framework that performs efficient multi-objective search over a broad space of agentic and non-agentic RAG configurations. Using Bayesian Optimization, syftr discovers Pareto-optimal flows that jointly optimize task accuracy and cost. A novel early-stopping mechanism further improves efficiency by pruning clearly suboptimal candidates. Across multiple RAG benchmarks, syftr finds flows which are on average approximately 9 times cheaper while preserving most of the accuracy of the most accurate flows on the Pareto-frontier. Furthermore, syftr's ability to design and optimize allows integrating new modules, making it even easier and faster to realize high-performing generative AI pipelines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
