触发掉队者：混合专家大型语言模型上的负载劫持
Trigger the Straggler: Load Hijack on Mixture-of-Experts LLMs
AI总结：
该研究提出针对混合专家大型语言模型的负载劫持攻击，通过修改路由器权重，使触发令牌集中于特定GPU的专家，造成该GPU成为掉队者，验证了中毒路由器可作为触发器控制的调度器，推动相关审计。
AI中文摘要：
专家并行（EP）是一种常见策略，通过在多个GPU间分配专家来部署大型混合专家（MoE）模型。路由器的决策决定了每个令牌由哪些专家处理，以及哪些GPU执行相应工作。这一过程在部署调度中暴露了供应链攻击面。我们提出负载劫持（Load Hijack）：恶意模型提供者仅修改检查点的路由器权重，分发中毒的检查点并保留私有触发器。当触发器出现时，中毒路由器会将令牌到专家的分配集中在同一GPU上的专家，产生的负载使该GPU成为掉队者，迫使对等设备等待，而普通输入的路由仍接近干净参考。我们发现这种条件行为难以实现，因为奖励触发输入上目标专家使用的目标函数也会使普通输入的路由偏向相同专家。为解决此冲突，负载劫持采用三阶段优化流程，在触发输入上产生强的依赖触发器的集中效果，同时保持普通输入路由接近干净参考。在三个MoE家族和四个语料库上，负载劫持将92.3%至95.6%的触发令牌分配导向目标专家。在实时EP部署中，触发流量产生的首令牌时间是普通流量下的1.43倍，吞吐量为普通流量下的0.86倍。这些结果表明，中毒路由器可作为触发器控制的设备调度器，推动对路由和运行时负载的检查点审计。
英文摘要：
Expert parallelism (EP) is a common strategy for serving large Mixture-of-Experts (MoE) models across multiple GPUs by distributing experts among devices. Router decisions then determine both which experts process each token and which GPUs execute the resulting work. This procedure exposes a supply-chain attack surface in the serving schedule. We introduce Load Hijack, in which a malicious model provider modifies only a checkpoint's router weights, distributes the poisoned checkpoint, and retains a private trigger. When the trigger appears, the poisoned router concentrates token-to-expert assignments on experts co-located on one GPU. The resulting load makes that GPU a straggler and forces peer devices to wait, while routing on ordinary inputs remains near the clean reference. We find this conditional behavior difficult to achieve because an objective that rewards target-expert use on triggered inputs can also bias ordinary-input routing toward the same experts. To resolve this conflict, Load Hijack employs a three-stage optimization procedure that produces strong trigger-dependent concentration while keeping ordinary-input routing close to the clean reference. Across three MoE families and four corpora, Load Hijack directs 92.3% to 95.6% of triggered token assignments to the target experts. In live EP serving, triggered traffic produces 1.43x the time-to-first-token and 0.86x the throughput measured under ordinary traffic. These results show that poisoned routers can act as trigger-controlled device schedulers and motivate checkpoint audits of routing and runtime load.
