arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2604.04825cs.CLcs.AI

可解释性作为常识推理:人类成功,大语言模型却不成功

Plausibility as Commonsense Reasoning: Humans Succeed, Large Language Models Do not

University of Chicago(芝加哥大学)

Sercan Karakaş

AI总结:

研究探讨大语言模型是否能像人类一样在歧义解析中结合世界知识与句法结构。通过土耳其前置定语从句附着歧义,发现人类表现出显著的可解释性效应,而大语言模型则表现不一致。

摘要:

大语言模型在许多语言任务中表现出色,但尚不清楚它们是否能像人类一样在歧义解析中结合世界知识与句法结构。我们通过土耳其前置定语从句附着歧义测试这一问题,其中相同表层字符串允许高附着(HA)或低附着(LA)。我们构建了保持句法配置固定且两种解析都语用上可能的歧义项目,同时按事件可解释性等级选择性地偏好高附着 vs. 低附着。对比通过独立规范评分验证。在一项速度受限的强制选择理解实验中,人类表现出显著的可解释性效应。然后,我们评估了土耳其语和多语言LLM在平行偏好基础上的设置中,通过每token对数概率比较匹配的HA/LA延续。在所有模型中,可解释性驱动的转变是弱、不稳定或反转的。结果表明,在测试模型中,可解释性信息无法像人类判断那样可靠地指导附着偏好,并突显土耳其RC附着作为跨语言诊断工具的价值,超越了广泛基准。

英文摘要:

Large language models achieve strong performance on many language tasks, yet it remains unclear whether they integrate world knowledge with syntactic structure in a human-like, structure-sensitive way during ambiguity resolution. We test this question in Turkish prenominal relative-clause attachment ambiguities, where the same surface string permits high attachment (HA) or low attachment (LA). We construct ambiguous items that keep the syntactic configuration fixed and ensure both parses remain pragmatically possible, while graded event plausibility selectively favors High Attachment vs.\ Low Attachment. The contrasts are validated with independent norming ratings. In a speeded forced-choice comprehension experiment, humans show a large, correctly directed plausibility effect. We then evaluate Turkish and multilingual LLMs in a parallel preference-based setup that compares matched HA/LA continuations via mean per-token log-probability. Across models, plausibility-driven shifts are weak, unstable, or reversed. The results suggest that, in the tested models, plausibility information does not guide attachment preferences as reliably as it does in human judgments, and they highlight Turkish RC attachment as a useful cross-linguistic diagnostic beyond broad benchmarks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑