用于高效且准确的多语言语音翻译的新型寄生双尺度建模
Novel Parasitic Dual-Scale Modeling for Efficient and Accurate Multilingual Speech Translation
AI总结：
本文提出寄生双尺度方法，结合投机采样、模型压缩与知识蒸馏，在Whisper Medium基础上构建whisperM2M并集成KVSPN模块，实现多语言语音翻译在六种语言上的SOTA性能及最高2.6倍加速。
摘要：
语音到文本翻译的最新进展推动了能够同时处理多个语言对的多语言模型的发展。然而，这些统一模型通常参数规模较大，使得在推理效率与性能之间取得平衡变得困难，尤其是在本地部署场景中。我们提出了一种创新的寄生双尺度方法，该方法将增强的投机采样方法与模型压缩和知识蒸馏技术相结合。在Whisper Medium模型的基础上，我们将其增强为用于多语言语音翻译的whisperM2M，并集成了我们新颖的KVSPN模块，在六种流行语言上取得了最先进（SOTA）性能，同时提高了推理效率。KVSPN实现了40%的加速，且BLEU分数无退化。结合蒸馏方法，它相比原始Whisper Medium实现了2.6倍的加速，并具有更优越的性能。
英文摘要：
Recent advancements in speech-to-text translation have led to the development of multilingual models capable of handling multiple language pairs simultaneously. However, these unified models often suffer from large parameter sizes, making it challenging to balance inference efficiency and performance, particularly in local deployment scenarios. We propose an innovative Parasitic Dual-Scale Approach, which combines an enhanced speculative sampling method with model compression and knowledge distillation techniques. Building on the Whisper Medium model, we enhance it for multilingual speech translation into whisperM2M, and integrate our novel KVSPN module, achieving state-of-the-art (SOTA) performance across six popular languages with improved inference efficiency. KVSPN enables a 40\% speedup with no BLEU score degradation. Combined with distillation methods, it represents a 2.6$\times$ speedup over the original Whisper Medium with superior performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
