寻找最佳平衡点:在推理时LLM反思中权衡质量、成本与速度
Finding the Sweet Spot: Trading Quality, Cost, and Speed During Inference-Time LLM Reflection
AI总结:
本文系统比较了自我反思与预算调整在数学推理和翻译任务中的表现,发现反思有效性因领域而异,数学推理性能提升最高达220%,并在真实营销内容本地化系统中验证了领域特定评估的重要性。
摘要:
随着大型语言模型(LLMs)的持续发展,实践者在无需重新训练模型的情况下,面临着越来越多增强推理时性能的选择,包括预算调整和多步骤技术(如自我反思)。虽然这些方法提高了输出质量,但它们也带来了准确性、成本和延迟之间复杂的权衡,这些权衡在不同领域中的表现尚不明确。本文系统地比较了自我反思和预算调整在数学推理和翻译任务中的表现。我们评估了包括Anthropic Claude、Amazon Nova和Mistral系列在内的多个主流LLM,以及其他模型,在不同反思深度和计算预算下,推导出帕累托最优性能前沿。我们的分析揭示了自我反思有效性在不同领域存在显著差异,在数学推理中性能提升最高可达220%。我们进一步研究了反思轮次深度和反馈机制质量如何影响不同模型家族的性能。为了在真实场景中验证我们的发现,我们在Zalando旗下的Lounge部署了一个增强自我反思的营销内容本地化系统,该系统显示出与市场相关的有效性,从而强调了在部署这些技术时进行领域特定评估的重要性。我们的结果为在特定领域和资源约束下选择最优推理策略提供了可操作的指导。我们开源了自我反思实现,以确保可复现性,代码见https://github.com/aws-samples/sample-genai-reflection-for-bedrock。
英文摘要:
As Large Language Models (LLMs) continue to evolve, practitioners face increasing options for enhancing inference-time performance without model retraining, including budget tuning and multi-step techniques like self-reflection. While these methods improve output quality, they create complex trade-offs among accuracy, cost, and latency that remain poorly understood across different domains. This paper systematically compares self-reflection and budget tuning across mathematical reasoning and translation tasks. We evaluate prominent LLMs, including Anthropic Claude, Amazon Nova, and Mistral families, along with other models under varying reflection depths and compute budgets to derive Pareto optimal performance frontiers. Our analysis reveals substantial domain dependent variation in self-reflection effectiveness, with performance gains up to 220\% in mathematical reasoning. We further investigate how reflection round depth and feedback mechanism quality influence performance across model families. To validate our findings in a real-world setting, we deploy a self-reflection enhanced marketing content localisation system at Lounge by Zalando, where it shows market-dependent effectiveness, reinforcing the importance of domain specific evaluation when deploying these techniques. Our results provide actionable guidance for selecting optimal inference strategies given specific domains and resource constraints. We open source our self-reflection implementation for reproducibility at https://github.com/aws-samples/sample-genai-reflection-for-bedrock.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。