自我训练与Few-Shot具体化--Few-Shot中的教师指导与学生NLU
Self-training with Few-shot Rationalization: Teacher Explanations Aid Student in Few-shot NLU
摘要：
虽然预训练的语言模型在几个自然语言理解任务中获得了最先进的性能，但它们在决策过程中非常不透明。虽然最近的一些工作专注于通过突出文本中的突出概念作为理由或理由来合理化神经预测，但它们依赖于数千个标记的训练样本用于任务标签以及注释的每个实例的理由。对于许多任务来说，获得这种广泛的大规模注释是不可行的。为此，我们开发了一个多任务的教师-学生框架的基础上自我训练语言模型有限的任务特定的标签和理由，明智的样本选择，从信息丰富的伪标记的例子1。我们研究了构成良好理论基础的几个特征，并证明了神经模型的性能可以通过使其意识到其合理化预测来显着提高，特别是在低资源环境中。在多个基准数据集上的实验证明了该方法的有效性。
英文摘要：
While pre-trained language models have obtained state-of-the-art performance for several natural language understanding tasks, they are quite opaque in terms of their decision-making process. While some recent works focus on rationalizing neural predictions by highlighting salient concepts in the text as justifications or rationales, they rely on thousands of labeled training examples for both task labels as well as an-notated rationales for every instance. Such extensive large-scale annotations are infeasible to obtain for many tasks. To this end, we develop a multi-task teacher-student framework based on self-training language models with limited task-specific labels and rationales, and judicious sample selection to learn from informative pseudo-labeled examples1. We study several characteristics of what constitutes a good rationale and demonstrate that the neural model performance can be significantly improved by making it aware of its rationalized predictions, particularly in low-resource settings. Extensive experiments in several bench-mark datasets demonstrate the effectiveness of our approach.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
