用于鲁棒语言模型的提示扰动一致性学习
Prompt Perturbation Consistency Learning for Robust Language Models
AI总结：
本文提出提示扰动一致性学习（PPCL），通过正则化干净与扰动样本损失差异，提升大语言模型在意图分类和槽位填充任务中对输入扰动的鲁棒性，并显著恢复性能下降。
摘要：
大语言模型（LLMs）在诸多自然语言处理任务上展现了令人印象深刻的性能，例如问答和文本摘要。然而，它们在序列标注任务上的表现，如意旨分类和槽位填充（IC-SF）——这是个人助理系统中的核心组成部分——显著落后于判别式模型。此外，关于LLMs对输入提示中各种扰动的鲁棒性，目前缺乏实质性的研究。本文的贡献有三方面。首先，我们表明对足够大的LLMs进行微调可以产生与判别式模型相当的IC-SF性能。其次，我们系统分析了这些微调模型因三种不同但相关的输入扰动类型——同音异义词、同义词和改写——而导致的性能下降。最后，我们提出了一种高效的缓解方法，即提示扰动一致性学习（PPCL），其原理是通过正则化干净样本与扰动样本损失之间的差异来工作。我们的实验表明，PPCL在IC和SF任务上分别平均恢复了59%和69%的性能下降。此外，PPCL在使用少十倍增强数据样本的情况下，优于数据增强方法。
英文摘要：
Large language models (LLMs) have demonstrated impressive performance on a number of natural language processing tasks, such as question answering and text summarization. However, their performance on sequence labeling tasks such as intent classification and slot filling (IC-SF), which is a central component in personal assistant systems, lags significantly behind discriminative models. Furthermore, there is a lack of substantive research on the robustness of LLMs to various perturbations in the input prompts. The contributions of this paper are three-fold. First, we show that fine-tuning sufficiently large LLMs can produce IC-SF performance comparable to discriminative models. Next, we systematically analyze the performance deterioration of those fine-tuned models due to three distinct yet relevant types of input perturbations - oronyms, synonyms, and paraphrasing. Finally, we propose an efficient mitigation approach, Prompt Perturbation Consistency Learning (PPCL), which works by regularizing the divergence between losses from clean and perturbed samples. Our experiments demonstrate that PPCL can recover on average 59% and 69% of the performance drop for IC and SF tasks, respectively. Furthermore, PPCL beats the data augmentation approach while using ten times fewer augmented data samples.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
