Optimas: 通过全局对齐的局部奖励优化复合AI系统
Optimas: Optimizing Compound AI Systems with Globally Aligned Local Rewards
AI总结:
Optimas通过全局对齐的局部奖励优化复合AI系统,有效提升复合系统的性能。
摘要:
复合AI系统集成了多种组件,如大型语言模型、专用工具和传统机器学习模型,越来越多地被用于解决复杂现实任务。然而,优化复合系统仍然具有挑战性,因为它们的非可微结构和组件之间的配置类型多样,包括提示、超参数和模型参数。为了解决这一挑战,我们提出了Optimas,一个统一的框架,用于有效优化复合系统。Optimas的核心思想是在每个组件中维护一个局部奖励函数(LRF),每个都满足局部-全局对齐属性,即每个组件的局部奖励与全局系统性能相关联。在每次迭代中,Optimas高效地调整LRFs以维持这一属性,同时同时最大化每个组件的局部奖励。这种方法使异构配置能够独立更新,使用指定的优化方法,同时确保局部改进始终导致性能提升。我们展示了在五个现实中的广泛评估复合系统,证明Optimas在平均改进11.92%的情况下优于强大的基线,提供了一种通用且有效的改进复合系统的方法。我们的网站是https://optimas.stanford.edu。
英文摘要:
Compound AI systems integrating multiple components, such as Large Language Models, specialized tools, and traditional machine learning models, are increasingly deployed to solve complex real-world tasks. However, optimizing compound systems remains challenging due to their non-differentiable structures and diverse configuration types across components, including prompts, hyperparameters, and model parameters. To address this challenge, we propose Optimas, a unified framework for effective optimization of compound systems. The core idea of Optimas is to maintain one Local Reward Function (LRF) per component, each satisfying a local-global alignment property, i.e., each component's local reward correlates with the global system performance. In each iteration, Optimas efficiently adapts the LRFs to maintain this property while simultaneously maximizing each component's local reward. This approach enables independent updates of heterogeneous configurations using the designated optimization method, while ensuring that local improvements consistently lead to performance gains. We present extensive evaluations across five real-world compound systems to demonstrate that Optimas outperforms strong baselines by an average improvement of 11.92%, offering a general and effective approach for improving compound systems. Our website is at https://optimas.stanford.edu.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。