KbSD: 面向智能体搜索中行为校准的知识边界感知自蒸馏
KbSD: Knowledge Boundary aware Self-Distillation for Behavioral Calibration in Agentic Search
AI总结：
针对智能体搜索中知识边界校准的奖励稀疏问题，提出KbSD框架，通过密集令牌级监督、稀疏结果奖励和象限自适应优化，利用提示增强教师模型生成校准推理演示，显著提升任务准确率并减少幻觉。
摘要：
智能体搜索赋予大语言模型动态检索能力，但现有的强化学习方法在知识边界校准——决定何时信任参数记忆、何时依赖检索证据以及何时弃权——方面仍受限于奖励稀疏性。二元奖励可以惩罚不良结果，但对跨不同知识状态做出校准决策所需的推理过程提供的指导很少。为了解决这个问题，我们提出了KbSD（知识边界自蒸馏）框架，通过密集的令牌级监督、结果级稀疏奖励和象限自适应优化来应对这一限制。KbSD构建了一个提示增强的教师模型，其在架构上与学生模型相同，接收明确的知识边界信号——包括参数确定性、检索质量和真实答案——以生成校准的推理演示。这种信息不对称的自蒸馏实现了密集监督，而无需更大的外部模型。为了进一步考虑跨知识状态的异构推理分布，我们引入了象限自适应蒸馏目标：针对集中整合的反向KL，针对多样化拒绝的前向KL，以及针对需要精度和覆盖度的非对称象限的帕累托最优双向KL。在多个基准上的实验表明，KbSD在强基线上持续提高了任务准确率和幻觉缓解，最大的提升出现在具有挑战性的象限中，这些象限中稀疏奖励的信息量最少。
英文摘要：
Agentic search equips large language models with dynamic retrieval abilities, but existing reinforcement learning methods remain limited by reward sparsity in knowledge boundary calibration -- deciding when to trust parametric memory, when to rely on retrieved evidence, and when to abstain. Binary rewards can penalize undesirable outcomes, but provide little guidance on the reasoning process required to make calibrated decisions across different knowledge states. To address this, we propose KbSD (Knowledge boundary Self-Distillation), a framework that tackles this limitation through dense token-level supervision, outcome-level sparse rewards, and quadrant-adaptive optimization. KbSD constructs a hint-augmented teacher, architecturally identical to the student, that receives explicit knowledge boundary signals -- including parametric certainty, retrieval quality, and ground-truth answers -- to generate calibrated reasoning demonstrations. This information-asymmetric self-distillation enables dense supervision without requiring a larger external model. To further account for the heterogeneous reasoning distributions across knowledge states, we introduce a quadrant-adaptive distillation objective: reverse KL for concentrated integration, forward KL for diverse refusal, and Pareto-optimal bidirectional KL for asymmetric quadrants requiring both precision and coverage. Experiments on multiple benchmarks show that KbSD consistently improves both task accuracy and hallucination mitigation over strong baselines, with the largest gains appearing in the challenging quadrants where sparse rewards are least informative.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
