从专有模型到开源模型：通过智能体搜索中的多智能体协议蒸馏弥合分布差距
From Proprietary to Open-Source: Bridging the Distribution Gap via Multi-Agent Protocol Distillation in Agentic Search
- Beijing Institute of Technology（北京理工大学）
- East China Normal University（华东师范大学）
- University of Science and Technology of China（中国科学技术大学）
- Tsinghua University（清华大学）
- University of Chinese Academy of Sciences（中国科学院大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
研究旨在弥合专有与开源模型分布差距，提出多智能体协议蒸馏（MAPD）框架，利用结构化协议作中间表示，结合蒸馏与强化学习，在问答基准测试中表现出色，有效减轻学生策略问题，优于其他竞争方法。
AI中文摘要：
智能体搜索使大语言模型能通过多步推理与检索解决知识密集型任务，但基于结果的强化学习优化时监督稀疏。知识蒸馏可提供更密集指导，先进专有模型是有潜力的教师。然而，从专有模型蒸馏存在异构问题，传统方法受限。为此提出多智能体协议蒸馏（MAPD），它使用结构化、风格归一化协议作为中间表示。离线多智能体系统分解查询、检索证据、修复失败搜索并转换探索轨迹为JSON协议。训练时，协议仅提供给学生策略的特权分支，其令牌分布提供密集蒸馏信号。在七个问答基准测试中，MAPD持续优于竞争方法，在Qwen3-1.7B上平均成功率达39.4%，在Qwen3-4B上达44.4%，且能有效减轻学生策略的风格漂移和冗长退化问题。
英文摘要：
Agentic search enables large language models to solve knowledge-intensive tasks by interleaving multi-step reasoning with retrieval, yet optimizing this with outcome-based reinforcement learning (RL) provides only sparse supervision. Knowledge distillation can supply denser guidance, and advanced proprietary models with their strong reasoning capabilities are promising teachers. While distilling from proprietary models can densify this supervisory signal, conventional logit-matching is precluded by hidden logits and mismatched tokenizers, whereas raw natural language trajectory imitation transfers superficial stylistic artifacts rather than core reasoning competence. To address the heterogeneous distillation problem and bridge the distribution gap, we propose Multi-Agent Protocol Distillation (MAPD), a joint distillation and RL framework uses a structured, style-normalized protocol as an intermediate representation. An offline multi-agent system (MAS) decomposes each query, retrieves supporting evidence, repairs failed searches, and converts the resulting exploration trace into a JSON protocol containing the task type, reasoning plan, and extractive grounding facts. During training, the protocol is provided only to a privileged branch of the student policy, whose token distributions furnish a dense distillation signal alongside the sparse RL objective. Extensive evaluations across seven QA benchmarks demonstrate that MAPD consistently outperforms competitive distillation and RL, achieving average success rates of 39.4\% on Qwen3-1.7B and 44.4\% on Qwen3-4B. Crucially, the framework generalizes robustly across diverse proprietary teachers while effectively mitigating the student policy from style drift and verbosity degeneration.
