如果可能,那么可接受?理解大型语言模型中的条件可接受性判断
If Probable, Then Acceptable? Understanding Conditional Acceptability Judgments in Large Language Models
AI总结:
研究探讨了大型语言模型在评估条件语句'若A则B'的可接受性时如何结合条件概率与语义相关性,发现模型对这两因素的敏感度因架构和提示策略而异,且比人类更不一致。
摘要:
条件可接受性指的是人们对条件语句的合理性感知。它在沟通和推理中起重要作用,影响个体如何解释蕴含、评估论点并基于假设情境做决策。当人类评估'若A则B'的可接受性时,其判断受两个主要因素影响:B在A下的条件概率和A在B下的语义相关性(即A是否有意义地支持B)。尽管已有研究探讨了大型语言模型(LLMs)如何推断条件语句,但尚不清楚这些模型如何判断此类语句的可接受性。为填补这一空白,我们对不同模型家族、大小和提示策略下的LLMs的条件可接受性判断进行了全面研究。使用线性混合效应模型和方差分析检验,我们发现模型对条件概率和语义相关性敏感,但敏感度因架构和提示方式而异。与人类数据的比较表明,尽管LLMs整合了概率和语义线索,但其一致性不如人类。值得注意的是,较大模型并不一定更接近人类判断。
英文摘要:
Conditional acceptability refers to how plausible a conditional statement is perceived to be. It plays an important role in communication and reasoning, as it influences how individuals interpret implications, assess arguments, and make decisions based on hypothetical scenarios. When humans evaluate how acceptable a conditional "If A, then B" is, their judgments are influenced by two main factors: the $\textit{conditional probability}$ of $B$ given $A$, and the $\textit{semantic relevance}$ of the antecedent $A$ given the consequent $B$ (i.e., whether $A$ meaningfully supports $B$). While prior work has examined how large language models (LLMs) draw inferences about conditional statements, it remains unclear how these models judge the $\textit{acceptability}$ of such statements. To address this gap, we present a comprehensive study of LLMs' conditional acceptability judgments across different model families, sizes, and prompting strategies. Using linear mixed-effects models and ANOVA tests, we find that models are sensitive to both conditional probability and semantic relevance$\unicode{x2014}$though to varying degrees depending on architecture and prompting style. A comparison with human data reveals that while LLMs incorporate probabilistic and semantic cues, they do so less consistently than humans. Notably, larger models do not necessarily align more closely with human judgments.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。