基于不确定性两级选择优化大型语言模型调用
Optimising Calls to Large Language Models with Uncertainty-Based Two-Tier Selection
AI总结:
本研究提出仅基于小型LLM生成不确定性的两级选择策略来优化调用,在27个实验设置中的25个上优于现有方法,实现成本与性能的最佳平衡。
摘要:
在有限预算下工作的研究人员和从业者面临成本与性能之间的权衡困境。具有挑战性的决策往往集中在是使用性能更好的大型LLM还是成本更低的小型LLM上。这推动了近期关于LLM调用优化的研究。要么采用级联策略,即依次调用较小的LLM或两者都调用;要么采用路由策略,即仅调用一个模型。这两种场景都依赖于一个决策标准,该标准通常由一个额外的神经模型实现。在这项工作中,我们提出了一种更简单的解决方案;我们仅使用小型LLM生成结果的不确定性作为决策标准。我们将我们的方法与级联和路由策略进行了比较,使用了三对不同规模的预训练小型和大型LLM,在九个不同的任务上,并与需要额外神经模型的方法进行了对比。我们的实验表明,这种简单的解决方案能够最优地平衡成本与性能,在27个实验设置中的25个上优于现有方法。
英文摘要:
Researchers and practitioners operating on a limited budget face the cost-performance trade-off dilemma. The challenging decision often centers on whether to use a large LLM with better performance or a smaller one with reduced costs. This has motivated recent research in the optimisation of LLM calls. Either a cascading strategy is used, where a smaller LLM or both are called sequentially, or a routing strategy is used, where only one model is ever called. Both scenarios are dependent on a decision criterion which is typically implemented by an extra neural model. In this work, we propose a simpler solution; we use only the uncertainty of the generations of the small LLM as the decision criterion. We compare our approach with both cascading and routing strategies using three different pairs of pre-trained small and large LLMs, on nine different tasks and against approaches that require an additional neural model. Our experiments reveal this simple solution optimally balances cost and performance, outperforming existing methods on 25 out of 27 experimental setups.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。