小型语言模型代理实现高效高质量的知识挖掘
A Tale of LLMs and Induced Small Proxies: Scalable Small Language Models for Knowledge Mining
AI总结:
提出Falconer框架,结合大语言模型的代理推理与轻量级代理模型,通过规划与标注实现可扩展的知识挖掘,在保持指令遵循精度的同时降低90%推理成本并加速20倍以上。
摘要:
深度研究(Deep Research)的核心是知识挖掘,即根据用户指令从海量非结构化文本中提取结构化信息的任务。大语言模型(LLMs)擅长解释此类指令,但大规模部署成本过高;而传统的分类器和提取器流水线虽然高效,但脆弱且无法泛化到新任务。我们提出Falconer,一种协作框架,将LLM的代理推理与轻量级代理模型相结合,用于可扩展的知识挖掘。在Falconer中,LLM作为规划者,将用户指令分解为可执行的流水线,并作为标注者,生成监督信号以训练小型代理。该框架将分类和提取统一为两个原子操作:get label和get span,使得单个指令遵循模型能够替代多个任务特定组件。为了评估Falconer孵化的代理模型与人类和大模型提供的标注之间的一致性,我们构建了涵盖规划和端到端执行的新基准。实验表明,Falconer在指令遵循精度上接近最先进的LLM,同时将推理成本降低高达90%,并将大规模知识挖掘加速20倍以上,为深度研究提供了高效且可扩展的基础。
英文摘要:
At the core of Deep Research is knowledge mining, the task of extracting structured information from massive unstructured text in response to user instructions. Large language models (LLMs) excel at interpreting such instructions but are prohibitively expensive to deploy at scale, while traditional pipelines of classifiers and extractors remain efficient yet brittle and unable to generalize to new tasks. We introduce Falconer, a collaborative framework that combines the agentic reasoning of LLMs with lightweight proxy models for scalable knowledge mining. In Falconer, LLMs act as planners, decomposing user instructions into executable pipelines, and as annotators, generating supervision to train small proxies. The framework unifies classification and extraction into two atomic operations, get label and get span, enabling a single instruction-following model to replace multiple task-specific components. To evaluate the consistency between proxy models incubated by Falconer and annotations provided by humans and large models, we construct new benchmarks covering both planning and end-to-end execution. Experiments show that Falconer closely matches state-of-the-art LLMs in instruction-following accuracy while reducing inference cost by up to 90% and accelerating large-scale knowledge mining by more than 20x, offering an efficient and scalable foundation for Deep Research.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。