arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新，休息一下吧，周末愉快~~
arXiv 2602.02204cs.DC
收藏

vLLM-Omni: 任何到任何多模态模型的完全解耦服务

vLLM-Omni: Fully Disaggregated Serving for Any-to-Any Multimodal Models

AI Framework and Data Technology Lab, Huawei（华为人工智能框架与数据技术实验室）；The Chinese University of Hong Kong（香港中文大学）；Institute of Software, Chinese Academy of Sciences（中国科学院软件研究所）；Sun Yat-sen University（中山大学）
Peiqi Yin, Jiangyun Zhu, Han Gao, Chenguang Zheng, Yongxiang Huang, Taichang Zhou, Ruirui Yang, Weizhi Liu, Weiqing Chen, Canlin Guo, Didan Deng, Zifeng Mo, Con… 展开作者

Peiqi Yin, Jiangyun Zhu, Han Gao, Chenguang Zheng, Yongxiang Huang, Taichang Zhou, Ruirui Yang, Weizhi Liu, Weiqing Chen, Canlin Guo, Didan Deng, Zifeng Mo, Cong Wang, James Cheng, Roger Wang, Hongsheng Liu

备注：
12 pages, 8 figures
阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

vLLM-Omni通过解耦服务系统优化多模态模型的高效服务，显著提升作业完成时间效率。

摘要：

任何到任何多模态模型，能够同时处理文本、图像、视频和音频，代表了多模态AI的重要进展。然而，其复杂的架构（通常结合多个自回归LLM、扩散Transformer和其他专用组件）对高效的模型服务提出了重大挑战。现有的服务系统主要是针对单一范式，如自回归LLM用于文本生成或扩散Transformer用于视觉生成。它们不支持任何到任何的流水线，涉及多个相互连接的模型组件。因此，开发者必须手动处理跨阶段交互，导致性能大幅下降。我们提出了vLLM-Omni，一个专为任何到任何模型设计的完全解耦服务系统。vLLM-Omni具有一个新颖的阶段抽象，使用户能够将复杂的任何到任何架构分解为相互连接的阶段，表示为图。并且具有一个解耦的阶段执行后端，优化跨阶段的资源利用和吞吐量。每个阶段由LLM或扩散引擎独立服务，具有每阶段请求批处理、灵活的GPU分配和统一的跨阶段连接器用于数据路由。实验结果表明，与基线方法相比，vLLM-Omni将作业完成时间（JCT）减少了高达91.4%。代码可在https://github.com/vllm-project/vllm-omni上公开获取。

英文摘要：

Any-to-any multimodal models that jointly handle text, images, video, and audio represent a significant advance in multimodal AI. However, their complex architectures (typically combining multiple autoregressive LLMs, diffusion transformers, and other specialized components) pose substantial challenges for efficient model serving. Existing serving systems are mainly tailored to a single paradigm, such as autoregressive LLMs for text generation or diffusion transformers for visual generation. They lack support for any-to-any pipelines that involve multiple interconnected model components. As a result, developers must manually handle cross-stage interactions, leading to huge performance degradation. We present vLLM-Omni, a fully disaggregated serving system for any-to-any models. vLLM-Omni features a novel stage abstraction that enables users to decompose complex any-to-any architectures into interconnected stages represented as a graph, and a disaggregated stage execution backend that optimizes resource utilization and throughput across stages. Each stage is independently served by an LLM or diffusion engine with per-stage request batching, flexible GPU allocation, and unified inter-stage connectors for data routing. Experimental results demonstrate that vLLM-Omni reduces job completion time (JCT) by up to 91.4% compared to baseline methods. The code is public available at https://github.com/vllm-project/vllm-omni.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑