STAR: 基于相似性引导的教师辅助细化方法用于超小功能调用模型
STAR: Similarity-guided Teacher-Assisted Refinement for Super-Tiny Function Calling Models
AI总结：
STAR通过约束知识蒸馏和基于相似性的强化学习，有效将大语言模型能力转移到超小模型，实现复杂功能调用任务的高性能表现。
摘要：
大语言模型（LLMs）在功能调用中的普及对于创建高级AI代理至关重要，但其大规模限制了广泛应用，需要将其能力转移到更小的模型中。然而，现有方法常常面临过拟合、训练不稳定、多解任务中二元奖励效果差以及技术协同困难的问题。我们引入STAR：基于相似性的教师辅助细化方法，一种新的整体框架，有效将LLMs的能力转移到超小模型中。STAR包含两个核心技术创新：（1）约束知识蒸馏（CKD），一种训练目标，通过增强top-k前向KL散度来抑制自信错误预测，确保训练稳定性的同时保留下游RL的探索能力。STAR在统一的训练课程中整体协同这些策略，使超小模型在复杂功能调用任务中取得优异表现；（2）基于相似性的强化学习（Sim-RL），一种强化学习机制，引入基于细粒度的相似性奖励。这为更好的策略优化提供了稳健、连续且丰富的信号，通过评估生成输出与地面真实之间的相似性。在具有挑战性和知名的基准测试中进行了大量实验，证明了我们方法的有效性。我们的STAR模型在其规模类别中建立了SOTA，显著优于基线。令人瞩目的是，我们的0.6B STAR模型在所有1B以下的开放模型中取得了最佳性能，超越了甚至一些更大规模的知名开放模型。STAR展示了一种将LLMs能力蒸馏到超小模型的训练框架，为强大、可访问且高效的AI代理铺平了道路。
英文摘要：
The proliferation of Large Language Models (LLMs) in function calling is pivotal for creating advanced AI agents, yet their large scale hinders widespread adoption, necessitating transferring their capabilities into smaller ones. However, existing paradigms are often plagued by overfitting, training instability, ineffective binary rewards for multi-solution tasks, and the difficulty of synergizing techniques. We introduce STAR: Similarity-guided Teacher-Assisted Refinement, a novel holistic framework that effectively transfers LLMs' capabilities to super-tiny models. STAR consists of two core technical innovations: (1) Constrained Knowledge Distillation (CKD), a training objective that augments top-k forward KL divergence to suppress confidently incorrect predictions, ensuring training stability while preserving exploration capacity for downstream RL. STAR holistically synergizes these strategies within a cohesive training curriculum, enabling super-tiny models to achieve exceptional performance on complex function calling tasks; (2) Similarity-guided RL (Sim-RL), a RL mechanism that introduces a fine-grained, similarity-based reward. This provides a robust, continuous, and rich signal for better policy optimization by evaluating the similarity between generated outputs and the ground truth. Extensive experiments on challenging and renowned benchmarks demonstrate the effectiveness of our method. Our STAR models establish SOTA in their size classes, significantly outperforming baselines. Remarkably, our 0.6B STAR model achieves the best performance among all open models under 1B, surpassing even several well-known open models at a larger scale. STAR demonstrates a training framework that distills capabilities of LLMs into super-tiny models, paving the way for powerful, accessible, and efficient AI agents.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
