大型语言模型中的条件推理与模态推理
Conditional and Modal Reasoning in Large Language Models
AI总结：
本文评估29个大型语言模型在条件句和认识模态词推理上的表现，发现即使使用零样本思维链提示，表现最好的模型仍存在基本逻辑错误、判断不一致及与人类判断不符等问题，揭示了当前LLM在基本逻辑推理上的差距。
摘要：
大型语言模型（LLMs）的推理能力是人工智能和认知科学领域日益增多的研究主题。在本文中，我们探究了29个LLM在多大程度上能够区分逻辑正确的推论与逻辑谬误。我们重点关注涉及条件句（例如，“如果Ann有一张Q，那么Bob有一张J”）和认识模态词（例如，“Ann可能有A”，“Bob必须有K”）的推理模式。这些推论一直引起逻辑学家、哲学家和语言学家的特别关注，因为它们在人类推理远端可能性的基本能力中发挥着核心作用。因此，在这些推论上评估LLM，对于回答LLM的推理能力在多大程度上与人类相匹配这一问题具有高度相关性。我们测试的所有LLM在条件句或模态词上都犯有一些基本错误，尽管零样本思维链提示有助于它们减少错误。即使是表现最好的LLM，在模态推理中也会犯基本错误，在涉及认识模态词和条件句的推理模式中表现出逻辑不一致的判断，并且对复杂条件推论给出的答案与报告的人类判断不符。这些结果突显了当今LLM在基本逻辑推理方面的差距。
英文摘要：
The reasoning abilities of large language models (LLMs) are the topic of a growing body of research in AI and cognitive science. In this paper, we probe the extent to which twenty-nine LLMs are able to distinguish logically correct inferences from logically fallacious ones. We focus on inference patterns involving conditionals (e.g., 'If Ann has a queen, then Bob has a jack') and epistemic modals (e.g., 'Ann might have an ace', 'Bob must have a king'). These inferences have been of special interest to logicians, philosophers, and linguists, since they play a central role in the fundamental human ability to reason about distal possibilities. Assessing LLMs on these inferences is thus highly relevant to the question of how much the reasoning abilities of LLMs match those of humans. All the LLMs we tested make some basic mistakes with conditionals or modals, though zero-shot chain-of-thought prompting helps them make fewer mistakes. Even the best performing LLMs make basic errors in modal reasoning, display logically inconsistent judgments across inference patterns involving epistemic modals and conditionals, and give answers about complex conditional inferences that do not match reported human judgments. These results highlight gaps in basic logical reasoning in today's LLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
