通过黑盒在线调优提升大语言模型性能:为可信AI添加系统规格至事实表的案例
Improving LLM Performance Through Black-Box Online Tuning: A Case for Adding System Specs to Factsheets for Trusted AI
AI总结:
本文提出通过黑盒在线调优提升大语言模型性能,并探讨将系统性能和可持续性指标整合到事实表中对可信AI的重要性。
摘要:
本文提出了一种新颖的黑盒在线控制器,仅使用端到端的短段测量,不依赖内部仪器,并通过爬坡法最大化goodput,即满足服务级别目标的请求数量的吞吐量。我们提供了实证证据证明这种设计是稳固的。以LLM服务的这一进步为例,我们随后讨论了将系统性能和可持续性指标整合到事实表中对于采用AI系统的组织的重要性。
英文摘要:
In this paper, we present a novel black-box online controller that uses only end-to-end measurements over short segments, without internal instrumentation, and hill climbing to maximize goodput, defined as the throughput of requests that satisfy the service-level objective. We provide empirical evidence that this design is well-founded. Using this advance in LLM serving as a concrete example, we then discuss the importance of integrating system performance and sustainability metrics into Factsheets for organizations adopting AI systems.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。