软提示调优用于跨语言迁移：少即是多
Soft Prompt Tuning for Cross-Lingual Transfer: When Less is More
AI总结：
本文研究软提示调优（SPT）在跨语言迁移中的应用，通过冻结模型参数仅训练软提示，证明参数效率能提升向语言距离较远语言的迁移性能，并探讨提示长度和重新参数化等因素的影响。
摘要：
软提示调优（SPT）是一种参数高效的方法，通过在预训练语言模型（PLM）的输入层插入可学习的嵌入（即软提示）来使PLM适应特定任务，而无需修改其参数。本文研究了SPT在跨语言迁移中的潜力。与以往关于SPT用于跨语言迁移的研究通常同时微调软提示和模型参数不同，我们遵循SPT的原始意图，保持模型参数冻结，仅训练软提示。这不仅降低了全模型微调的计算成本和存储开销，而且我们证明了SPT固有的这种参数效率能够提升向语言距离较远的语言的跨语言迁移性能。此外，我们探讨了与提示相关的不同因素（如提示长度或其重新参数化）如何影响跨语言迁移性能。
英文摘要：
Soft Prompt Tuning (SPT) is a parameter-efficient method for adapting pre-trained language models (PLMs) to specific tasks by inserting learnable embeddings, or soft prompts, at the input layer of the PLM, without modifying its parameters. This paper investigates the potential of SPT for cross-lingual transfer. Unlike previous studies on SPT for cross-lingual transfer that often fine-tune both the soft prompt and the model parameters, we adhere to the original intent of SPT by keeping the model parameters frozen and only training the soft prompt. This does not only reduce the computational cost and storage overhead of full-model fine-tuning, but we also demonstrate that this very parameter efficiency intrinsic to SPT can enhance cross-lingual transfer performance to linguistically distant languages. Moreover, we explore how different factors related to the prompt, such as the length or its reparameterization, affect cross-lingual transfer performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
