DISCO：利用大型语言模型蒸馏反事实数据
DISCO: Distilling Counterfactuals with Large Language Models
AI总结：
本文提出 DISCO 流程，用大型通用语言模型生成短语扰动并由任务教师模型过滤蒸馏，大规模获得高质量反事实数据，显著提升小型 NLI 模型的鲁棒性、跨分布泛化和反事实一致性。
摘要：
使用反事实增强数据训练的模型能够学习任务因果结构的表示，从而实现稳健的泛化。然而，对大多数任务而言，高质量反事实数据十分稀缺，且难以大规模生成。通过众包获得的此类数据通常在规模和多样性上受限；而使用监督方法生成时，扩展到新的反事实维度又会带来高昂的计算成本。在本研究中，我们提出 DISCO（DIStilled COunterfactual Data，蒸馏反事实数据），这是一种用于大规模自动生成高质量反事实数据的新方法。DISCO 设计提示词，利用大型通用语言模型生成短语层面的扰动；随后，一个面向特定任务的教师模型过滤这些生成结果，以蒸馏出高质量反事实数据。尽管该流程与任务无关，我们仍将其应用于自然语言推理（NLI）任务，并发现：在 NLI 压力测试等具有挑战性的评估中，与未使用数据增强训练的模型相比，使用 DISCO 生成的反事实数据训练的相对较小的学生模型稳健性更高（绝对提升 6%），且跨分布泛化能力更好（提升 2%）。此外，在三个评估集上，经 DISCO 增强的模型在反事实样本对之间的一致性提高 10%，表明 DISCO 增强使模型能够更可靠地学习因果表示。我们的代码库位于：https://github.com/eric11eca/disco
英文摘要：
Models trained with counterfactually augmented data learn representations of the causal structure of tasks, enabling robust generalization. However, high-quality counterfactual data is scarce for most tasks and not easily generated at scale. When crowdsourced, such data is typically limited in scale and diversity; when generated using supervised methods, it is computationally expensive to extend to new counterfactual dimensions. In this work, we introduce DISCO (DIStilled COunterfactual Data), a new method for automatically generating high quality counterfactual data at scale. DISCO engineers prompts to generate phrasal perturbations with a large general language model. Then, a task-specific teacher model filters these generations to distill high-quality counterfactual data. While task-agnostic, we apply our pipeline to the task of natural language inference (NLI) and find that on challenging evaluations such as the NLI stress test, comparatively smaller student models trained with DISCO generated counterfactuals are more robust (6% absolute) and generalize better across distributions (2%) compared to models trained without data augmentation. Furthermore, DISCO augmented models are 10% more consistent between counterfactual pairs on three evaluation sets, demonstrating that DISCO augmentation enables models to more reliably learn causal representations. Our repository is available at: https://github.com/eric11eca/disco
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
