SkewRoute：基于检索上下文分数偏斜的无训练知识图谱检索增强生成LLM路由
SkewRoute: Training-Free LLM Routing for Knowledge Graph Retrieval-Augmented Generation via Score Skewness of Retrieved Context
AI总结：
针对知识图谱检索增强生成中的高推理成本问题，提出首个无训练的即插即用LLM路由框架SkewRoute，利用检索分数分布的偏斜性高效平衡性能与成本，路由有效性提升超3倍且运行时间降至0.001倍以下。
摘要：
大型语言模型在许多任务上表现出色，但在部署期间经常产生高昂的推理成本。为了缓解幻觉，许多系统使用知识图谱来增强检索增强生成（KG-RAG）。然而，大量检索到的知识上下文进一步增加了这些推理成本。平衡性能和成本的一个有前途的解决方案是LLM路由，它将简单查询导向较小的LLM，将复杂查询导向较大的LLM。然而，目前RAG尚无专用的路由方法，且现有的基于训练的路由器由于需要大量训练数据，在扩展到该领域时面临挑战。我们观察到，检索评分器产生的分数分布与查询难度强烈相关。基于此，我们提出了一个极其简单却有效的路由框架，这是首个专门为KG-RAG设计的框架，以即插即用的方式高效平衡性能和成本。与现有方法相比，它提供了超过3倍的更高路由有效性，同时将运行时间减少到不到0.001倍。我们的代码可在https://github.com/hrwang00/SkewRoute获取。
英文摘要：
Large language models excel at many tasks but often incur high inference costs during deployment. To mitigate hallucination, many systems use a knowledge graph to enhance retrieval-augmented generation (KG-RAG). However, the large amount of retrieved knowledge contexts increase these inference costs further. A promising solution to balance performance and cost is LLM routing, which directs simple queries to smaller LLMs and complex ones to larger LLMs. However, no dedicated routing methods currently exist for RAG, and existing training-based routers face challenges scaling to this domain due to the need for extensive training data. We observe that the score distributions produced by the retrieval scorer strongly correlate with query difficulty. Based on this, we propose an extremely simple yet effective routing framework, the first specifically designed for KG-RAG that efficiently balances performance and cost in a plug-and-play manner. It delivers over 3x higher routing effectiveness while reducing runtime to less than 0.001x compared to existing methods. Our code is available at https://github.com/hrwang00/SkewRoute.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
