arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1609.00344cs.CV
收藏

深度学习人类思维用于自动视觉分类

Deep Learning Human Mind for Automated Visual Classification

University of Central Florida(中佛罗里达大学)

Concetto Spampinato, Simone Palazzo, Isaak Kavasidis, Daniela Giordano, Mubarak Shah, Nasim Souly

摘要:

如果我们能够有效地阅读思想,并将人类的视觉能力转移到计算机视觉方法中,那会怎么样?在本文中,我们的目标是通过开发第一个由人脑信号驱动的视觉对象分类器来解决这个问题。特别是,我们采用脑电数据诱发的视觉对象刺激结合递归神经网络(RNN)学习一个有区别的大脑活动流形的视觉类别。之后,我们训练一个基于卷积神经网络(CNN)的回归器,将图像投影到学习的流形上,从而有效地允许机器采用基于人脑的特征进行自动视觉分类。我们使用32通道EEG记录7名受试者的大脑活动,同时查看40个ImageNet对象类的图像。所提出的基于RNN的使用脑信号区分对象类别的方法达到了约40%的平均准确度,这优于试图学习EEG视觉对象表示的现有方法。至于自动对象分类,我们的人脑驱动方法在ImageNet和CalTech 101上获得了与强大的CNN模型相当的竞争性能,从而证明了其分类和泛化能力。这给了我们一个真正的希望,人类的思想确实可以被读取并转移到机器上。

英文摘要:

What if we could effectively read the mind and transfer human visual capabilities to computer vision methods? In this paper, we aim at addressing this question by developing the first visual object classifier driven by human brain signals. In particular, we employ EEG data evoked by visual object stimuli combined with Recurrent Neural Networks (RNN) to learn a discriminative brain activity manifold of visual categories. Afterwards, we train a Convolutional Neural Network (CNN)-based regressor to project images onto the learned manifold, thus effectively allowing machines to employ human brain-based features for automated visual classification. We use a 32-channel EEG to record brain activity of seven subjects while looking at images of 40 ImageNet object classes. The proposed RNN based approach for discriminating object classes using brain signals reaches an average accuracy of about 40%, which outperforms existing methods attempting to learn EEG visual object representations. As for automated object categorization, our human brain-driven approach obtains competitive performance, comparable to those achieved by powerful CNN models, both on ImageNet and CalTech 101, thus demonstrating its classification and generalization capabilities. This gives us a real hope that, indeed, human mind can be read and transferred to machines.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑