arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.02386cs.AIcs.IRcs.LG
收藏

信任由设计:基于技能配置的透明、成本感知的LLM路由

Trust by Design: Skill Profiles for Transparent, Cost-Aware LLM Routing

Georgia Institute of Technology(佐治亚理工学院)

Mika Okamoto, Ansel Kaplan Erol, Glenn Matlin

AI总结:

BELLA框架通过基于技能的可解释方法,为LLM路由提供透明、成本感知的最优模型选择方案,帮助从业者在性能与成本间做出合理权衡。

摘要:

大型语言模型(LLM)从业者应该如何在不浪费金钱的情况下选择合适的模型?我们介绍了BELLA(通过自动技能配置进行预算高效LLM选择)框架,该框架通过可解释的基于技能的模型选择推荐最优的LLM选择。标准基准报告聚合指标,这些指标掩盖了任务所需的具体能力以及更便宜的模型是否足够。BELLA通过三个阶段解决这一差距:(1)分解LLM输出并提取所需的细粒度技能,使用基于批评者的配置;(2)将技能聚类为结构化能力矩阵;(3)多目标优化以选择正确的模型,以在尊重预算约束的同时最大化性能。BELLA为推荐提供自然语言理由,提供当前黑箱路由系统所缺乏的透明度。我们描述了框架架构,将其置于LLM路由和评估的背景下,并讨论其在金融推理中的应用,作为代表领域,该领域表现出多样化的技能需求和模型间的成本差异。我们的框架使从业者能够为部署LLM做出原则性和成本-性能权衡。

英文摘要:

How should Large Language Model (LLM) practitioners select the right model for a task without wasting money? We introduce BELLA (Budget-Efficient LLM Selection via Automated skill-profiling), a framework that recommends optimal LLM selection for tasks through interpretable skill-based model selection. Standard benchmarks report aggregate metrics that obscure which specific capabilities a task requires and whether a cheaper model could suffice. BELLA addresses this gap through three stages: (1) decomposing LLM outputs and extract the granular skills required by using critic-based profiling, (2) clustering skills into structured capability matrices, and (3) multi-objective optimization to select the right models to maximize performance while respecting budget constraints. BELLA provides natural-language rationale for recommendations, providing transparency that current black-box routing systems lack. We describe the framework architecture, situate it within the landscape of LLM routing and evaluation, and discuss its application to financial reasoning as a representative domain exhibiting diverse skill requirements and cost-variation across models. Our framework enables practitioners to make principled and cost-performance trade-offs for deploying LLMs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑