无需嵌入的路由：基于正则表达式的快速可解释路由
Routing Without Embeddings: Fast And Interpretable Routing With Regular Expressions
- Rice University（莱斯大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
本文提出REGEXROUTE，利用稀疏自编码器发现可解释的正则表达式特征，替代神经编码实现快速可解释的LLM路由，在四个基准上达到与强基线相当的准确率。
AI中文摘要：
大语言模型（LLM）路由器通常依赖神经查询嵌入，预期更大的编码器能更好地捕捉查询意图和难度。然而，将Qwen2.5编码器从0.5B参数扩展到72B参数，路由准确率仅略有提升（图1b），这表明小编码器可能已能捕捉路由所需的查询属性。因此，我们研究哪些属性重要，以及这些属性是否可以直接从文本中提取而无需神经编码器。我们提出REGEXROUTE，一种利用稀疏自编码器（SAEs）发现可解释的正则表达式（regex）特征的流程。利用无标签文本，LLM将分组的SAE潜在变量描述转化为正则表达式提取器，并对其进行优化以匹配潜在激活模式。这些提取器为轻量级路由头提供数值特征，从而在推理时消除神经编码（图1a）。在四个基准测试中，一组固定的128个特征实现了76.43%的平均路由准确率，与最强神经文本编码器基线的76.41%相当，同时延迟更小且鲁棒性强。这些发现确立了显式、可解释的文本特征作为设计和理解LLM路由器的实用基础。
英文摘要：
Large Language Model (LLM) routers commonly rely on neural query embeddings, with larger encoders expected to better capture query intent and difficulty. Yet scaling Qwen2.5 encoders from 0.5B to 72B parameters brings little improvement in routing accuracy (Figure 1b), suggesting that small encoders may already capture the query properties needed for routing. We therefore investigate which properties matter and whether they can be extracted directly from text without a neural encoder. We introduce REGEXROUTE, a pipeline that uses sparse autoencoders (SAEs) to discover interpretable regular-expression (regex) features. Using unlabeled text, an LLM turns descriptions of grouped SAE latents into regex extractors and refines them to match latent activation patterns. These extractors supply numerical features to a lightweight routing head, eliminating neural encoding at inference (Figure 1a). Across four benchmarks, one fixed set of 128 features achieves 76.43% average routing accuracy, comparable to 76.41% for the strongest neural text encoder baseline, with much smaller latency and strong robustness. These findings establish explicit, interpretable text features as a practical basis for designing and understanding LLM routers.
