逻辑常识问答：逻辑常识推理的基准测试
LOGICAL-COMMONSENSEQA: A Benchmark for Logical Commonsense Reasoning
AI总结：
本文提出LOGICAL-COMMONSENSEQA基准，通过逻辑运算符评估常识推理，发现模型在否定类问题上表现下降，揭示了推理局限性。
摘要：
常识推理通常涉及评估多个可能解释而非单一答案，但多数基准依赖单标签评估，无法判断陈述是否共同合理、互斥或共同不合理。我们引入LOGICAL-COMMONSENSEQA基准，将常识推理重新定义为通过可能性级别运算符（AND、OR和NEITHER/NOR）在原子陈述对上的逻辑组合。在零样本、少样本和链式思维提示下评估指令调优、推理专用和微调模型，发现模型在合取推理上表现良好，但在否定类问题上表现急剧下降。LOGICAL-COMMONSENSEQA揭示了基本推理限制，并提供了一个受控框架以推进组合常识推理。
英文摘要：
Commonsense reasoning often involves evaluating multiple plausible interpretations rather than selecting a single atomic answer, yet most benchmarks rely on single-label evaluation, obscuring whether statements are jointly plausible, mutually exclusive, or jointly implausible. We introduce LOGICAL-COMMONSENSEQA, a benchmark that reframes commonsense reasoning as logical composition over pairs of atomic statements using plausibility-level operators (AND, OR and NEITHER/NOR). Evaluating instruction-tuned, reasoning-specialized, and fine-tuned models under zero-shot, few-shot, and chain-of-thought prompting, we find that while models perform reasonably on conjunctive and moderately on disjunctive reasoning, performance degrades sharply on negation-based questions. LOGICAL-COMMONSENSEQA exposes fundamental reasoning limitations and provides a controlled framework for advancing compositional commonsense reasoning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
