可靠性导向的多教师蒸馏用于低资源抽象摘要
Reliability Gated Multi-Teacher Distillation for Low Resource Abstractive Summarization
AI总结：
本文研究了从可靠性角度出发的多教师知识蒸馏，提出EWAD和CPDP机制，通过实验发现logit级KD效果最佳，复杂蒸馏提升短摘要语义相似性但影响长摘要，跨语言伪标签KD在压缩率下保持ROUGE L分数，揭示了单判 pipelines 的校准偏差。
摘要：
我们从可靠性角度研究了低资源抽象摘要中的多教师知识蒸馏。我们引入EWAD（熵加权一致性感知蒸馏），一种基于教师间一致性在token级别路由监督的机制，并引入CPDP（容量比例发散保持），一种对学生成绩相对于异质教师的几何约束。在两个孟加拉语数据集、13个孟加拉语T5消融实验和八个Qwen2.5实验中，我们发现logit级KD提供最可靠的增益，而更复杂的蒸馏提高了短摘要的语义相似性但影响长摘要。跨语言伪标签KD在十种语言中保持71-122%的教师ROUGE L分数在3.2倍压缩下。人类验证的多判LLM评估进一步揭示了单判流程中的校准偏差。总体而言，我们的结果表明，可靠性感知的蒸馏有助于确定多教师监督何时提高摘要质量，以及数据扩展何时超过损失工程。
英文摘要：
We study multiteacher knowledge distillation for low resource abstractive summarization from a reliability aware perspective. We introduce EWAD (Entropy Weighted Agreement Aware Distillation), a token level mechanism that routes supervision between teacher distillation and gold supervision based on inter teacher agreement, and CPDP (Capacity Proportional Divergence Preservation), a geometric constraint on the student position relative to heterogeneous teachers. Across two Bangla datasets, 13 BanglaT5 ablations, and eight Qwen2.5 experiments, we find that logit level KD provides the most reliable gains, while more complex distillation improves semantic similarity for short summaries but degrades longer outputs. Cross lingual pseudo label KD across ten languages retains 71-122 percent of teacher ROUGE L at 3.2x compression. A human validated multi judge LLM evaluation further reveals calibration bias in single judge pipelines. Overall, our results show that reliability aware distillation helps characterize when multi teacher supervision improves summarization and when data scaling outweighs loss engineering.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
