面向生命周期的知识蒸馏评估：机器翻译中的环境影响与翻译质量权衡
Is Knowledge Distillation Actually Greener? A Case Study in Machine Translation
AI总结：
本研究评估了机器翻译中知识蒸馏方法的环境影响与翻译质量的权衡，发现蒸馏开销和推理成本在不同部署规模下主导总足迹，词级蒸馏通常更优。
摘要：
知识蒸馏（KD）是一种将较大系统（教师）压缩成较小系统（学生）的工具。在机器翻译中，研究通常只报告学生模型的翻译质量，而省略了执行KD的计算复杂性，这使得在计算约束下难以选择众多可用的KD方法。在本研究中，我们通过同时考虑翻译质量和计算成本评估了代表性KD方法。我们将计算成本表示为碳足迹，使用机器学习生命周期评估（MLCA）工具。该评估考虑了KD模型生命周期（教师训练、蒸馏和推理）中的运行时操作排放和摊还的硬件生产成本。我们发现（i）在小部署量下，蒸馏开销主导总足迹，（ii）在大规模下推理主导，使KD仅在任务依赖的使用阈值以上才具有优势，（iii）词级蒸馏通常比序列级蒸馏提供更有利的足迹-质量权衡。我们的协议为在显式质量和计算约束下选择KD方法提供了可重复的指导。
英文摘要：
Knowledge distillation (KD) is a technique to compress a larger teacher system into a smaller student. In machine translation, KD is commonly evaluated through translation quality and inference efficiency, without jointly accounting for the environmental costs of producing and deploying the distilled system. We evaluate representative KD methods both on bespoke MT models and LLMs, by considering both translation quality and computational cost, using the Machine Learning Life Cycle Assessment tool, which accounts for costs throughout the KD model life cycle. Our key finding is that the deployment volume required to amortize KD is serving-dependent and can shift by several orders of magnitude under batching. We include actionable guidance for selecting, developing, and evaluating KD methods under quality and compute-induced constraints.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
