arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.07462cs.AI
收藏

机器是否像人类一样失败?一种以人为中心的分布外谱系用于映射误差对齐

Do Machines Fail Like Humans? A Human-Centred Out-of-Distribution Spectrum for Mapping Error Alignment

Binxia Xu, Xiaoliang Luo, Luke Dickens, Robert M. Mok

AI总结:

本文提出以人为中心的分布外谱系,通过量化刺激偏离参考集的程度,揭示不同深度学习架构在近分布外和远分布外条件下的模型-人类对齐差异。

摘要:

确定AI系统是否像人类一样处理信息是认知科学和可信AI的核心问题。尽管现代AI模型在标准任务上可与人类准确匹配,但这种对等性不保证其底层决策策略与人类相似。通过使用误差对齐度量来比较人类和模型在失败时的表现,并在受扰或更具挑战性的刺激下变化,提供了一种更精细的模型-人类对齐表征路径。然而,现有分布外(OOD)分析在挑战性刺激上的局限性源于方法学选择:它们将OOD位移定义为相对于模型训练数据,或使用任意的扰动特定参数,缺乏与人类感知的对应关系,阻碍了原则性比较。我们提出了一种以人为中心的框架,将OOD程度重新定义为人类感知难度的光谱。通过量化一组刺激偏离未受扰参考集的程度,基于人类准确性,我们构建了一个OOD光谱,并识别出四个不同的感知挑战阶段。这种方法使在校准难度水平上进行原则性模型-人类比较成为可能。我们将其应用于物体识别,揭示了不同深度学习架构在不同阶段的模型-人类对齐排名和特征。视觉语言模型在近分布外和远分布外条件下最一致地与人类对齐,但卷积神经网络(CNNs)在近分布外条件下的对齐程度比视觉转换器(ViTs)更高,而ViTs在远分布外条件下比CNNs更对齐。我们的工作展示了考虑跨条件差异(如感知难度)对于原则性评估模型-人类对齐的重要性。

英文摘要:

Determining whether AI systems process information similarly to humans is central to cognitive science and trustworthy AI. While modern AI models can match human accuracy on standard tasks, such parity does not guarantee that their underlying decision-making strategies resemble those of humans. Assessing performance using error alignment metrics to compare how humans and models fail, and how this changes for distorted, or otherwise more challenging, stimuli, provides a viable pathway toward a finer characterization of model-human alignment. However, existing out-of-distribution (OOD) analyses for challenging stimuli are limited due to methodological choices: they define OOD shift relative to model training data or use arbitrary distortion-specific parameters with little correspondence to human perception, hindering principled comparisons. We propose a human-centred framework that redefines the degree of OOD as a spectrum of human perceptual difficulty. By quantifying how much a collection of stimuli deviates from an undistorted reference set based on human accuracy, we construct an OOD spectrum and identify four distinct regimes of perceptual challenge. This approach enables principled model-human comparisons at calibrated difficulty levels. We apply this framework to object recognition and reveal unique, regime-dependent model-human alignment rankings and profiles across deep learning architectures. Vision-language models are most consistently human aligned across near- and far-OOD conditions, but convolutional neural networks (CNNs) are more aligned than vision transformers (ViTs) for near-OOD and ViTs are more aligned than CNNs for far-OOD. Our work demonstrates the critical importance of accounting for cross-condition differences, such as perceptual difficulty, for a principled assessment of model-human alignment.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑