面向模型市场的成本感知混合专家协调机制

中文摘要 AI 辅助

现有模型市场通常将单个模型作为不可分割的单元进行交易和选择，限制了其利用异构专家之间互补性的能力。本文提出一种基于混合专家（MoE）的模型市场框架，将混合专家从模型级学习架构提升至市场级协调机制。在该框架中，经纪人使用门控网络协调多个异构专家并交付复合模型服务。我们对市场参与者、服务工作流、专家成本结构以及结合预测效用与异构执行成本的福利目标进行了形式化定义。随后推导了成本感知门控机制和市场感知训练目标，并引入成本调整后的收益分配规则，该规则根据已实现的专家参与情况和执行成本分配剩余收益。我们还确立了该分配规则的基本理论性质，包括预算平衡、参与单调性和成本敏感性。在15个表格和图像基准测试上，使用5个随机种子进行的实验采用了独立训练并冻结的神经网络和基于树的专家，以及由延迟推导的执行成本。MoE市场在全部15个数据集上实现了最高的平均福利，且在所有情况下的平均预期成本均低于标准MoE，同时保持了有竞争力的预测性能。分配实验进一步证明了其对专家参与和成本的系统敏感性，以及比精确夏普利值分配低得多的计算开销。这些结果表明，MoE可作为协作、成本感知且具有经济基础的模型市场的市场级协调原则。

英文摘要

Existing model marketplaces typically trade and select individual models as indivisible units, limiting their ability to exploit complementarities among heterogeneous experts. This paper proposes an MoE-based model market framework that lifts Mixture-of-Experts from a model-level learning architecture to a market-level coordination mechanism. In this framework, brokers use gating networks to coordinate multiple heterogeneous experts and deliver a composite model service. We formalize the market participants, service workflow, expert cost structure, and a welfare objective that combines predictive utility with heterogeneous execution costs. We then derive a cost-aware gating mechanism and market-aware training objective, and introduce a cost-adjusted revenue allocation rule that distributes residual revenue according to realized expert participation and execution cost. We also establish basic theoretical properties of the allocation rule, including budget balance, participation monotonicity, and cost sensitivity. Experiments over five random seeds on fifteen tabular and image benchmarks use independently trained and frozen neural and tree-based experts together with latency-derived execution costs. MoE Market achieves the highest mean welfare on all fifteen datasets and a lower mean expected cost than Standard MoE in every case, while maintaining competitive predictive performance. The allocation experiments further demonstrate systematic sensitivity to expert participation and cost, together with substantially lower computational overhead than exact Shapley allocation. These results suggest that MoE can serve as a market-level coordination principle for collaborative, cost-aware, and economically grounded model marketplaces.

发表机构

• University of Southampton（南安普顿大学）
• Aalborg University（奥尔堡大学）
• University of Nottingham（诺丁汉大学）

机构由 AI 辅助整理，请以论文原文为准。