FMplex: 用于服务可扩展基础模型的模型虚拟化
FMplex: Model Virtualization for Serving Extensible Foundation Models
AI总结：
提出FMplex系统，通过将基础模型作为虚拟化层实现多任务共享，结合批感知公平队列调度器，在7个基础模型和92个下游任务上降低延迟达80%，提升任务容量6倍。
摘要：
基础模型（FMs）越来越多地被用作语言、视觉、时间序列和多模态应用的下游任务骨干。然而，现有的模型服务系统将每个定制任务部署为独立的模型实例，从而复制了重型骨干，浪费了加速器内存，并失去了摊销批处理和加载成本的机会。本文提出了FMplex，一个将FM骨干视为部署共享的虚拟化层的服务系统。FMplex为每个任务提供一个虚拟基础模型（vFM），这是一个由共享物理FM支持的逻辑私有FM实例。这种抽象允许独立定制的任务共享一个骨干，同时保留任务特定的扩展、独立生命周期和任务级隔离。此外，我们提出了一种批感知公平队列调度器，该调度器结合了加权任务级共享以及跨共存任务的批内和批间批处理。我们实现了一个基于FMplex的服务栈，涵盖任务构建、共享感知部署和运行时执行。在7个FM骨干（16个变体）和92个下游任务上，FMplex相比空间分区延迟降低高达80%，相比尽力而为共置延迟降低33.3%，同时在集群规模上可托管多达6倍的任务。
英文摘要：
Foundation models (FMs) are increasingly used as backbones for downstream tasks across language, vision, time-series, and multimodal applications. Yet existing model-serving systems deploy each customized task as an independent model instance, thereby replicating heavyweight backbones, wasting accelerator memory, and losing opportunities to amortize batching and loading costs. This paper presents FMplex, a serving system that treats FM backbones as a virtualization substrate for deployment sharing. FMplex presents each task with a virtual foundation model (vFM), a logically private FM instance backed by a shared physical FM. This abstraction lets independently customized tasks share a backbone while preserving task-specific extensions, independent lifecycles, and task-level isolation. In addition, we propose a batch-aware fair-queueing scheduler that combines weighted task-level sharing with inter- and intra-task batching across colocated tasks. We implement a FMplex-based serving stack spanning task construction, sharing-aware deployment, and runtime execution. Across 7 FM backbones (16 variants) and 92 downstream tasks, FMplex reduces latency by up to 80% over spatial partitioning and 33.3% over best-effort co-location, while hosting up to 6x more tasks at cluster scale.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
