DistiLLM-2：一种提升LLM蒸馏的对比方法
DistiLLM-2: A Contrastive Approach Boosts the Distillation of LLMs
AI总结：
针对LLM蒸馏中损失函数与数据类型协同不足的问题，提出对比方法DistiLLM-2，通过增加教师响应似然并降低学生响应似然，在多任务中显著提升学生模型性能并支持多样化应用。
摘要：
尽管蒸馏在大型语言模型（LLM）中取得了成功，但先前的大多数工作对教师生成的数据和学生生成的数据应用了相同的损失函数。这些策略忽视了损失函数公式与数据类型之间的协同作用，导致学生模型的性能提升处于次优状态。为了解决这一问题，我们提出了DistiLLM-2，这是一种对比方法，它通过利用这种协同作用，同时增加教师响应的似然并降低学生响应的似然。我们的大量实验表明，DistiLLM-2不仅在包括指令遵循和代码生成在内的广泛任务中构建了高性能的学生模型，而且还支持多样化的应用，例如偏好对齐和视觉语言扩展。这些发现突显了对比方法通过在不同数据类型上有效地对齐教师和学生模型，来增强LLM蒸馏效能的潜力。
英文摘要：
Despite the success of distillation in large language models (LLMs), most prior work applies identical loss functions to both teacher- and student-generated data. These strategies overlook the synergy between loss formulations and data types, leading to a suboptimal performance boost in student models. To address this, we propose DistiLLM-2, a contrastive approach that simultaneously increases the likelihood of teacher responses and decreases that of student responses by harnessing this synergy. Our extensive experiments show that DistiLLM-2 not only builds high-performing student models across a wide range of tasks, including instruction-following and code generation, but also supports diverse applications, such as preference alignment and vision-language extensions. These findings highlight the potential of a contrastive approach to enhance the efficacy of LLM distillation by effectively aligning teacher and student models across varied data types.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
