发表机构
University of California, Irvine; University of Washington（加利福尼亚大学欧文分校; 华盛顿大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究针对LoRA专家混合模型的路由问题，提出VI-MoLE算法，通过信息价值分配替代不确定性路由，经实验验证其在多指标上优于现有路由机制。
AI 中文摘要
低秩适配专家混合模型（Mixtures of low-rank adaptation experts）通过将每个输入路由到适配器子集，提升了参数高效容量。近期的动态路由机制在路由或预测存在不确定性时激活更多专家，该规则默认将不确定性等同于有用的额外计算：不确定的样本可能包含未查询的专家补充证据，但也可能在所有专家达成一致后仍存在歧义。我们将路由问题建模为经认证的信息价值分配问题。VI-MoLE算法学习每个专家前缀后剩余的反事实风险，将这些预测转换为保留校准数据上的同步风险上界认证，并在单位成本下边际风险减少量最大的令牌-层动作上使用全局适配器预算。最终认证决定回答或弃权（不执行）。与不确定性门不同，该方法区分当前歧义、可恢复风险和残余风险。我们证明了同步认证有效性、递减认证增益下的最优贪心分配，以及价值估计误差下的分配遗憾。评估协议针对固定和动态MoE-LoRA路由机制，测试了匹配计算准确率、认证覆盖率、风险-覆盖率、分布偏移及尾部延迟。
英文摘要
Mixtures of low-rank adaptation experts increase parameter-efficient capacity by routing each input through a subset of adapters. Recent dynamic routers activate more experts when the router or prediction is uncertain. This rule silently equates uncertainty with useful additional computation: an uncertain example may contain complementary, unqueried expert evidence, but it may instead remain ambiguous after every expert agrees. We formulate routing as certified value-of-information allocation. VI-MoLE learns the counterfactual risk remaining after each expert prefix, converts these predictions into simultaneous upper-risk certificates on held-out calibration data, and spends a global adapter budget on the token--layer action with the largest certified marginal risk reduction per unit cost. A terminal certificate then decides whether to answer or abstain. Unlike an uncertainty gate, this procedure distinguishes present ambiguity from recoverable and residual risk. We prove simultaneous certificate validity, optimal greedy allocation under diminishing certified gains, and allocation regret under value-estimation error. The evaluation protocol tests matched-compute accuracy, certificate coverage, risk--coverage, distribution shift, and tail latency against fixed and dynamic MoE-LoRA routers.
