Drive-KD：自动驾驶中用于视觉语言模型的多教师知识蒸馏
Drive-KD: Multi-Teacher Distillation for VLMs in Autonomous Driving
AI总结：
本文提出Drive-KD框架，通过将自动驾驶分解为感知-推理-规划三元组，并利用知识蒸馏转移能力，构建了专用教师模型，并通过异构梯度投影缓解跨能力梯度冲突，验证了方法在不同模型家族和规模上的泛化能力，展示了蒸馏模型在自动驾驶任务中的优越性能。
摘要：
自动驾驶是一个重要且安全关键的任务，最近大型语言模型（LLM）和视觉语言模型（VLM）的进展为该领域提供了新的推理和规划可能性。然而，大模型需要大量GPU内存并表现出较高的推理延迟，而传统监督微调（SFT）往往难以弥补小模型的能力差距。为了解决这些限制，我们提出了Drive-KD，一个将自动驾驶分解为“感知-推理-规划”三元组并通过知识蒸馏转移这些能力的框架。我们识别出层特定的注意力作为蒸馏信号，构建出能够超越基线的专用单教师模型。此外，我们将这些单教师设置统一到多教师蒸馏框架中，并引入异构梯度投影以缓解跨能力梯度冲突。广泛的评估验证了我们的方法在不同模型家族和规模上的泛化能力。实验表明，我们的蒸馏InternVL3-1B模型在GPU内存方面仅为78B模型的约42倍，在吞吐量方面为11.4倍，且在DriveBench上整体性能优于同家族的预训练78B模型，并在规划维度上超越GPT-5.1，为高效自动驾驶VLMs提供了新的见解。
英文摘要：
Autonomous driving is an important and safety-critical task, and recent advances in LLMs/VLMs have opened new possibilities for reasoning and planning in this domain. However, large models demand substantial GPU memory and exhibit high inference latency, while conventional supervised fine-tuning (SFT) often struggles to bridge the capability gaps of small models. To address these limitations, we propose Drive-KD, a framework that decomposes autonomous driving into a "perception-reasoning-planning" triad and transfers these capabilities via knowledge distillation. We identify layer-specific attention as the distillation signal to construct capability-specific single-teacher models that outperform baselines. Moreover, we unify these single-teacher settings into a multi-teacher distillation framework and introduce asymmetric gradient projection to mitigate cross-capability gradient conflicts. Extensive evaluations validate the generalization of our method across diverse model families and scales. Experiments show that our distilled InternVL3-1B model, with ~42 times less GPU memory and ~11.4 times higher throughput, achieves better overall performance than the pretrained 78B model from the same family on DriveBench, and surpasses GPT-5.1 on the planning dimension, providing insights toward efficient autonomous driving VLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
