评估大型语言模型与NeuBAROCO：三段论推理能力与类人偏见
Evaluating Large Language Models with NeuBAROCO: Syllogistic Reasoning Ability and Human-like Biases
AI总结：
本研究通过NeuBAROCO数据集评估大型语言模型的三段论推理能力，发现其在处理信念偏见、转换错误和氛围效应三类类人偏见问题时表现更差。
摘要：
本文研究了当前大型语言模型在逻辑推理中是否表现出类似人类的偏见。具体而言，我们聚焦于三段论推理，这是人类演绎认知科学中一种被广泛研究的推理形式。为便于分析，我们引入了一个名为NeuBAROCO的数据集，该数据集最初设计用于评估人类在三段论推理中逻辑能力的心理实验。该数据集包含英语和日语的三段论推理。我们考察了人类三段论推理中观察到的三类偏见：信念偏见、转换错误和氛围效应。我们的研究结果表明，当前大型语言模型在处理涉及这三类偏见的问题时更为困难。
英文摘要：
This paper investigates whether current large language models exhibit biases in logical reasoning, similar to humans. Specifically, we focus on syllogistic reasoning, a well-studied form of inference in the cognitive science of human deduction. To facilitate our analysis, we introduce a dataset called NeuBAROCO, originally designed for psychological experiments that assess human logical abilities in syllogistic reasoning. The dataset consists of syllogistic inferences in both English and Japanese. We examine three types of biases observed in human syllogistic reasoning: belief biases, conversion errors, and atmosphere effects. Our findings demonstrate that current large language models struggle more with problems involving these three types of biases.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
