arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2306.01009cs.CLcs.AIcs.LG
收藏

考察生成式语言模型中演绎推理能力的涌现

Examining the Emergence of Deductive Reasoning in Generative Language Models

ETH Zürich(苏黎世联邦理工学院)

Peter Belcak, Luca A. Lanzendörfer, Roger Wattenhofer

AI总结:

本文评估多种参数规模的生成式 transformer-decoder 模型,发现演绎推理能力随规模增强,且除 GPT-3/GPT-3.5 外通常不受推理链长度影响。

摘要:

我们对生成式 transformer 模型根据所给前提进行演绎推理的能力开展了初步探究。我们观察到,来自不同训练设置的模型在性能上存在显著差异,并发现演绎推理能力会随规模提升而增强。此外,我们发现性能通常不会随着得出结论所需演绎链长度的增加而下降,但 OpenAI GPT-3 和 GPT-3.5 模型例外。本研究考察了多种 transformer-decoder 模型,参数规模从 1.17 亿到 1750 亿不等。

英文摘要:

We conduct a preliminary inquiry into the ability of generative transformer models to deductively reason from premises provided. We observe notable differences in the performance of models coming from different training setups and find that the deductive reasoning ability increases with scale. Further, we discover that the performance generally does not decrease with the length of the deductive chain needed to reach the conclusion, with the exception of OpenAI GPT-3 and GPT-3.5 models. Our study considers a wide variety of transformer-decoder models, ranging from 117 million to 175 billion parameters in size.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑