Tiny-Critic RAG:赋能代理回退的参数高效小型语言模型
Tiny-Critic RAG: Empowering Agentic Fallback with Parameter-Efficient Small Language Models
AI总结:
Tiny-Critic RAG通过参数高效的Small Language Model实现低延迟的二进制路由,有效降低代理部署成本。
摘要:
检索增强生成(RAG)通过将大型语言模型(LLMs)与事实性幻觉问题相结合,以减少事实性幻觉。最近的范式从静态流程转向模块化和代理RAG框架,使模型能够进行多跳推理或自我纠正。然而,当前的反思性RAG严重依赖于大规模LLM作为通用评估器。在高吞吐量系统中,仅为了二进制路由执行完整前向传递对于十亿参数模型来说会产生严重的计算冗余。此外,在自主代理场景中,不准确的检索会导致模型在虚假推理和冗余工具调用上消耗过多的token,增加时间到第一个token(TTFT)和成本。我们提出了Tiny-Critic RAG,通过低秩适应(LoRA)部署参数高效的Small Language Model(SLM)来解耦评估。Tiny-Critic作为确定性的守门人,采用受约束的解码和非思考推理模式,以实现超低延迟的二进制路由。在噪声注入数据集上的评估显示,Tiny-Critic RAG在路由准确性上与GPT-4o-mini相当,同时将延迟减少了数量级,建立了高度成本效益的代理部署范式。
英文摘要:
Retrieval-Augmented Generation (RAG) grounds Large Language Models (LLMs) to mitigate factual hallucinations. Recent paradigms shift from static pipelines to Modular and Agentic RAG frameworks, granting models autonomy for multi-hop reasoning or self-correction. However, current reflective RAG heavily relies on massive LLMs as universal evaluators. In high-throughput systems, executing complete forward passes for billion-parameter models merely for binary routing introduces severe computational redundancy. Furthermore, in autonomous agent scenarios, inaccurate retrieval causes models to expend excessive tokens on spurious reasoning and redundant tool calls, inflating Time-to-First-Token (TTFT) and costs. We propose Tiny-Critic RAG, decoupling evaluation by deploying a parameter-efficient Small Language Model (SLM) via Low-Rank Adaptation (LoRA). Acting as a deterministic gatekeeper, Tiny-Critic employs constrained decoding and non-thinking inference modes for ultra-low latency binary routing. Evaluations on noise-injected datasets demonstrate Tiny-Critic RAG achieves routing accuracy comparable to GPT-4o-mini while reducing latency by an order of magnitude, establishing a highly cost-effective paradigm for agent deployment.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。