arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2405.01601cs.CLcs.LG
收藏

高效的样本特定编码器扰动

Efficient Sample-Specific Encoder Perturbations

ALTA Institute, Department of Engineering, University of Cambridge(剑桥大学工程系ALTA研究所)

Yassir Fathullah, Mark J. F. Gales

AI总结:

本文提出用小型代理网络对冻结编码器-解码器基础模型进行样本特定扰动,以提升Flan-T5机器翻译COMET和Whisper语音识别WER性能,并验证了方法的鲁棒性与跨域扩展能力。

摘要:

编码器-解码器基础模型在一系列自回归序列任务上已展现出最先进的性能。本文提出一种简单且轻量的修改方式,用于根据感兴趣的特定属性控制此类系统的行为。具体而言,我们展示了一个小型代理网络可用于为冻结的基础模型逐样本寻找编码器输出的扰动,从而触发解码器生成改进的解码结果。本文探索了该框架的一个具体实现,重点提升Flan-T5在机器翻译上的COMET性能,以及Whisper基础模型在语音识别上的WER表现。实验结果表明,分别通过COMET和WER评估时,性能均呈现一致提升。此外,实验还表明这些代理网络对用于训练它们的数据的具体性质具有鲁棒性,并可扩展到其他领域。

英文摘要:

Encoder-decoder foundation models have displayed state-of-the-art performance on a range of autoregressive sequence tasks. This paper proposes a simple and lightweight modification to such systems to control the behaviour according to a specific attribute of interest. This paper proposes a novel inference-efficient approach to modifying the behaviour of an encoder-decoder system according to a specific attribute of interest. Specifically, we show that a small proxy network can be used to find a sample-by-sample perturbation of the encoder output of a frozen foundation model to trigger the decoder to generate improved decodings. This work explores a specific realization of this framework focused on improving the COMET performance of Flan-T5 on Machine Translation and the WER of Whisper foundation models on Speech Recognition. Results display consistent improvements in performance evaluated through COMET and WER respectively. Furthermore, experiments also show that the proxies are robust to the exact nature of the data used to train them and can extend to other domains.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑