从任务指令中学习的鲁棒性
Robustness of Learning from Task Instructions
AI总结：
本研究首次系统评估了预训练语言模型在指令驱动任务泛化中的鲁棒性，考察了指令被操纵、改写及简洁度变化对模型表现的影响。
摘要：
传统的监督学习主要针对单个任务，并需要在一大批特定任务的示例上进行训练。由于准备特定任务示例集的成本高昂，这种范式严重阻碍了任务泛化的发展。为了构建一个能够快速且轻松地泛化到新任务的系统，任务指令最近被采用为一种新兴的监督趋势。这些指令为模型提供任务定义，并允许模型根据指令和输入输出合适的答案。然而，任务指令通常以不同形式表达，可以从两个维度来解读：首先，一些指令是短句且面向预训练语言模型（PLM），例如提示（prompts），而其他指令是段落且面向人类，例如 Amazon MTurk 中的指令；其次，不同的终端用户极有可能用不同文本表达的指令来解释同一个任务。一个用于任务泛化的鲁棒系统应该能够处理任何新任务，而不受指令可变性的影响。然而，系统在处理指令驱动的任务泛化时的鲁棒性仍未被探索。本研究调查了当新任务的指令被（i）操纵、（ii）改写或（iii）处于不同简洁程度时系统的鲁棒性。据我们所知，这是首个系统性地研究 PLM 在受到具有不同可变因素的指令监督时其鲁棒性的工作。
英文摘要：
Traditional supervised learning mostly works on individual tasks and requires training on a large set of task-specific examples. This paradigm seriously hinders the development of task generalization since preparing a task-specific example set is costly. To build a system that can quickly and easily generalize to new tasks, task instructions have been adopted as an emerging trend of supervision recently. These instructions give the model the definition of the task and allow the model to output the appropriate answer based on the instructions and inputs. However, task instructions are often expressed in different forms, which can be interpreted from two threads: first, some instructions are short sentences and are pretrained language model (PLM) oriented, such as prompts, while other instructions are paragraphs and are human-oriented, such as those in Amazon MTurk; second, different end-users very likely explain the same task with instructions of different textual expressions. A robust system for task generalization should be able to handle any new tasks regardless of the variability of instructions. However, the system robustness in dealing with instruction-driven task generalization is still unexplored. This work investigates the system robustness when the instructions of new tasks are (i) manipulated, (ii) paraphrased, or (iii) from different levels of conciseness. To our knowledge, this is the first work that systematically studies how robust a PLM is when it is supervised by instructions with different factors of variability.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
