Thinker：通过多轮交互训练大语言模型（LLMs）进行分层思考以实现深度搜索
Thinker: Training LLMs in Hierarchical Thinking for Deep Search via Multi-Turn Interaction
AI总结：
针对现有LLM检索方法缺乏推理过程监督的问题，提出分层思考模型Thinker，通过多轮交互将复杂问题分解为子问题并双重表示，实现推理可监督，仅需少量样本即达基准性能，全量训练后显著优于现有方法。
摘要：
高效检索外部知识库和网页对于增强大语言模型（LLMs）的推理能力至关重要。此前训练LLMs利用外部检索器解决复杂问题的工作主要采用端到端强化学习，但这些方法忽略了对推理过程的监督，难以保证逻辑连贯性和严谨性。为解决这些局限性，我们提出Thinker，一种通过多轮交互进行深度搜索的分层思考模型，使推理过程可监督、可验证。它将复杂问题分解为可独立解决的子问题，每个子问题同时用自然语言和等效逻辑函数表示，以支持知识库和网页搜索；同时，子问题间的依赖关系通过这些逻辑函数作为参数传递，增强解题过程的逻辑连贯性。为避免不必要的外部搜索，我们进行知识边界判定，检查子问题是否在LLM的固有知识范围内，使其可直接回答。实验结果显示，仅用数百个训练样本，Thinker的性能就与现有基准相当；当扩展到完整训练集时，Thinker在不同数据集和模型规模上显著优于这些方法。源代码可在https://github.com/OpenSPG/KAG-Thinker获取。
英文摘要：
Efficient retrieval of external knowledge bases and web pages is crucial for enhancing the reasoning abilities of LLMs. Previous works on training LLMs to leverage external retrievers for solving complex problems have predominantly employed end-to-end reinforcement learning. However, these approaches neglect supervision over the reasoning process, making it difficult to guarantee logical coherence and rigor. To address these limitations, we propose Thinker, a hierarchical thinking model for deep search through multi-turn interaction, making the reasoning process supervisable and verifiable. It decomposes complex problems into independently solvable sub-problems, each dually represented in both natural language and an equivalent logical function to support knowledge base and web searches. Concurrently, dependencies between sub-problems are passed as parameters via these logical functions, enhancing the logical coherence of the problem-solving process. To avoid unnecessary external searches, we perform knowledge boundary determination to check if a sub-problem is within the LLM's intrinsic knowledge, allowing it to answer directly. Experimental results indicate that with as few as several hundred training samples, the performance of Thinker is competitive with established baselines. Furthermore, when scaled to the full training set, Thinker significantly outperforms these methods across various datasets and model sizes. The source code is available at https://github.com/OpenSPG/KAG-Thinker.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
