arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2009.07185cs.CLcs.AI
收藏

语言模型的批判性思维

Critical Thinking for Language Models

Karlsruhe Institute of Technology(卡尔斯鲁厄理工学院);Allen Institute for AI(艾伦人工智能研究所)

Gregor Betz, Christian Voigt, Kyle Richardson

摘要:

本文采取了第一步的批判性思维课程神经自回归语言模型。我们引入了一个综合语料库的演绎有效的论点,并产生人工论证文本训练和评估GPT-2。可以观察到显着的迁移学习效果:在三个简单的核心方案上训练模型,使其能够准确地完成不同的结论,更复杂类型的论点,也是如此。语言模型以正确的方式概括了核心论元模式。此外,我们获得了一致的和有前途的结果NLU基准。特别是,对参数方案的预训练将GLUE诊断的zero-shot准确度提高了15个百分点。研究结果表明,中介预训练的文本,提高基本的推理能力(如通常涵盖在批判性思维教科书)可能有助于语言模型获得广泛的推理技能。本文提出的综合性议论文是一个很有希望的起点,建立这样一个“批判性思维课程的语言模型。"

英文摘要:

This paper takes a first step towards a critical thinking curriculum for neural auto-regressive language models. We introduce a synthetic corpus of deductively valid arguments, and generate artificial argumentative texts to train and evaluate GPT-2. Significant transfer learning effects can be observed: Training a model on three simple core schemes allows it to accurately complete conclusions of different, and more complex types of arguments, too. The language models generalize the core argument schemes in a correct way. Moreover, we obtain consistent and promising results for NLU benchmarks. In particular, pre-training on the argument schemes raises zero-shot accuracy on the GLUE diagnostics by up to 15 percentage points. The findings suggest that intermediary pre-training on texts that exemplify basic reasoning abilities (such as typically covered in critical thinking textbooks) might help language models to acquire a broad range of reasoning skills. The synthetic argumentative texts presented in this paper are a promising starting point for building such a "critical thinking curriculum for language models."

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑