vLLM Semantic Router: 面向多模态混合模型的信号驱动决策路由
vLLM Semantic Router: Signal Driven Decision Routing for Mixture-of-Modality Models
AI总结：
提出vLLM Semantic Router框架，通过可组合信号编排（13种异构信号类型和布尔决策规则）实现多模态模型部署中的智能请求路由，支持不同场景的差异化策略配置。
摘要：
随着大语言模型在模态、能力和成本配置上的多样化，智能请求路由（即在推理时为每个查询选择合适模型）已成为关键的系统挑战。我们提出vLLM Semantic Router，一种面向多模态混合模型部署的信号驱动决策路由框架。该架构遵循两种互补的香农启发视角。在信息论机制中，信号提取通过从原始查询中提炼路由相关信息来降低“选择哪个模型？”的熵。在布尔代数机制中，决策引擎将信号条件组合成功能完备的路由策略。核心创新是可组合信号编排：13种异构信号类型（涵盖亚毫秒级启发式方法以及用于语义、安全性和模态的神经分类器）通过可配置的布尔决策规则组合成部署特定的路由策略，使得根本不同的场景（多云企业、隐私监管、成本优化）被表达为同一架构上的不同配置。匹配的决策通过13种选择算法驱动语义模型路由，而每个决策的插件链强制执行安全约束，包括三阶段HaluGate幻觉检测流水线和轻量级情景记忆系统（带ReflectionGate用于个性化多轮上下文）。一种类型化的神经符号DSL指定这些路由策略并将其编译到多个部署目标，实现无需代码更改的配置优先适配。这些组件共同表明，可组合信号编排使单一框架能够以差异化的成本、隐私和安全策略服务于多种部署场景。
英文摘要：
As large language models (LLMs) diversify across modalities, capabilities, and cost profiles, the problem of intelligent request routing: selecting the right model for each query at inference time, has become a critical systems challenge. We present vLLM Semantic Router, a signal-driven decision routing framework for Mixture-of-Modality (MoM) model deployments. The architecture follows two complementary Shannon-inspired views. In the information-theoretic regime, signal extraction reduces the entropy of "which model?" by distilling routing-relevant information from raw queries. In the Boolean-algebraic regime, the decision engine composes functionally complete routing policies from signal conditions. The central innovation is composable signal orchestration: thirteen heterogeneous signal types, spanning sub-millisecond heuristics and neural classifiers for semantics, safety, and modality, are composed through configurable Boolean decision rules into deployment-specific routing policies, so that fundamentally different scenarios (multi-cloud enterprise, privacy-regulated, cost-optimized) are expressed as different configurations over the same architecture. Matched decisions drive semantic model routing via thirteen selection algorithms, while per-decision plugin chains enforce safety constraints including a three-stage HaluGate hallucination detection pipeline and a lightweight episodic memory system with ReflectionGate for personalized multi-turn context. A typed neural-symbolic DSL specifies these routing policies and compiles them to multiple deployment targets, enabling configuration-first adaptation without code changes. Together, these components show that composable signal orchestration enables a single framework to serve diverse deployment scenarios with differentiated cost, privacy, and safety policies.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
