arXiv 2409.065092025-09-03cs.CVcs.AIcs.LG

跨抽象层级对齐机器与人类视觉表征

Aligning Machine and Human Visual Representations across Abstraction Levels

Google DeepMind Machine Learning Group（谷歌DeepMind机器学习组）；Technische Universität Berlin（技术大学柏林）；BIFOLD；Berlin Institute for the Foundations of Learning and Data（柏林学习与数据基础研究所）；Max Planck Institute for Human Cognitive and Brain Sciences（人类认知与脑科学Max Planck研究所）；Max Planck Institute for Human Development（人类发展Max Planck研究所）；TUD Dresden University of Technology（德累斯顿技术大学）；Anthropic；Department of Artificial Intelligence, Korea University（人工智能系，韩国大学）；Max Planck Institute for Informatics（信息Max Planck研究所）

Lukas Muttenthaler, Klaus Greff, Frieda Born, Bernhard Spitzer, Simon Kornblith, Michael C. Mozer, Klaus-Robert Müller, Thomas Unterthiner, Andrew K. Lampinen

备注：: 91 pages

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对视觉模型与人类概念知识层级结构错位的问题，研究通过教师模型迁移人类对齐结构微调视觉基础模型，既提升了模型与人类行为的一致性，也增强了泛化与分布外鲁棒性。

摘要：

深度神经网络已在广泛应用中取得成功，包括作为视觉任务中人类行为和神经表征的模型。然而，神经网络训练与人类学习存在根本性差异，且神经网络的泛化能力往往不如人类稳健，这引发了关于其底层表征相似性的疑问。现代学习系统要展现出更符合人类的行为，还缺少什么？我们指出了视觉模型与人类之间的一个关键错位：人类的概念知识是从精细到粗尺度的区分层级化组织的，而模型表征无法准确捕捉所有这些抽象层级。为解决这一错位，我们首先训练一个教师模型来模仿人类判断，随后通过微调，将其表征中符合人类认知的结构迁移出来，用以优化预训练的最先进视觉基础模型的表征。这些经人类对齐的模型在广泛的相似性任务中（包括一个涵盖多个语义抽象层级的人类判断新数据集），能更准确地逼近人类行为和不确定性。它们在多样的机器学习任务上也表现更优，提升了泛化能力和分布外鲁棒性。因此，为神经网络注入额外的人类知识可获得一种兼具优势的表征：既与人类认知判断更一致，又更具实用价值，从而为打造更稳健、可解释且符合人类认知的人工智能系统铺平道路。

英文摘要：

Deep neural networks have achieved success across a wide range of applications, including as models of human behavior and neural representations in vision tasks. However, neural network training and human learning differ in fundamental ways, and neural networks often fail to generalize as robustly as humans do raising questions regarding the similarity of their underlying representations. What is missing for modern learning systems to exhibit more human-aligned behavior? We highlight a key misalignment between vision models and humans: whereas human conceptual knowledge is hierarchically organized from fine- to coarse-scale distinctions, model representations do not accurately capture all these levels of abstraction. To address this misalignment, we first train a teacher model to imitate human judgments, then transfer human-aligned structure from its representations to refine the representations of pretrained state-of-the-art vision foundation models via finetuning. These human-aligned models more accurately approximate human behavior and uncertainty across a wide range of similarity tasks, including a new dataset of human judgments spanning multiple levels of semantic abstractions. They also perform better on a diverse set of machine learning tasks, increasing generalization and out-of-distribution robustness. Thus, infusing neural networks with additional human knowledge yields a best-of-both-worlds representation that is both more consistent with human cognitive judgments and more practically useful, thus paving the way toward more robust, interpretable, and human-aligned artificial intelligence systems.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。