来自不可信教师的诚实学生：从预训练语言模型学习可解释的问答流水线
Honest Students from Untrusted Teachers: Learning an Interpretable Question-Answering Pipeline from a Pretrained Language Model
AI总结：
提出markup-and-mask rationale风格，利用冻结的预训练语言模型作为不可信教师生成银标注数据，微调学生模型作为流水线瓶颈，在无标注下构建可信可解释问答系统。
摘要：
可解释的问答系统不仅应产生准确的答案，产生能够证明其推理合理并允许人类检查其工作的rationale。但什么样的rationale是有用的，我们如何训练系统来产生它们？我们提出了一种用于开放书问答的rationale新风格，称为markup-and-mask，它结合了抽取式和自由文本解释的方面。在markup阶段，通过自由文本markup增强文章，使每个句子能够在话语语境之外独立存在。在masking阶段，从被markup的文章中选择一个子跨度。为了在无标注的情况下训练系统产生markup-and-mask rationale，我们利用上下文学习。具体而言，我们通过向冻结的预训练语言模型发送一系列提示来生成银标注数据，该模型充当教师。然后，我们通过在导向正确答案的rationale子集上训练来微调学生模型。学生之所以诚实，是因为它是一个流水线：rationale充当文章和答案之间的瓶颈，而不可信的教师在没有此类约束的情况下运行。因此，我们提供了一种从端任务标注和冻结的预训练语言模型组合中构建可信流水线系统的新方法。
英文摘要：
Explainable question answering systems should produce not only accurate answers but also rationales that justify their reasoning and allow humans to check their work. But what sorts of rationales are useful and how can we train systems to produce them? We propose a new style of rationale for open-book question answering, called \emph{markup-and-mask}, which combines aspects of extractive and free-text explanations. In the markup phase, the passage is augmented with free-text markup that enables each sentence to stand on its own outside the discourse context. In the masking phase, a sub-span of the marked-up passage is selected. To train a system to produce markup-and-mask rationales without annotations, we leverage in-context learning. Specifically, we generate silver annotated data by sending a series of prompts to a frozen pretrained language model, which acts as a teacher. We then fine-tune a smaller student model by training on the subset of rationales that led to correct answers. The student is "honest" in the sense that it is a pipeline: the rationale acts as a bottleneck between the passage and the answer, while the "untrusted" teacher operates under no such constraints. Thus, we offer a new way to build trustworthy pipeline systems from a combination of end-task annotations and frozen pretrained language models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
