What if we could effectively read the mind and transfer human visual capabilities to computer vision methods? In this paper, we aim at addressing this question by developing the first visual object classifier driven by human brain signals. In particular, we employ EEG data evoked by visual object stimuli combined with Recurrent Neural Networks (RNN) to learn a discriminative brain activity manifold of visual categories. Afterwards, we train a Convolutional Neural Network (CNN)-based regressor to project images onto the learned manifold, thus effectively allowing machines to employ human brain-based features for automated visual classification. We use a 32-channel EEG to record brain activity of seven subjects while looking at images of 40 ImageNet object classes. The proposed RNN based approach for discriminating object classes using brain signals reaches an average accuracy of about 40%, which outperforms existing methods attempting to learn EEG visual object representations. As for automated object categorization, our human brain-driven approach obtains competitive performance, comparable to those achieved by powerful CNN models, both on ImageNet and CalTech 101, thus demonstrating its classification and generalization capabilities. This gives us a real hope that, indeed, human mind can be read and transferred to machines.
MindAdapter: Few-Shot Parameter-Efficient Residual Calibration of Cross-Subject Brain-to-Visual Decoding Models
Guangdong Institute of Intelligence Science and Technology(广东智能科学与技术研究院);Agency for Science, Technology and Research(科技研究局);New York University(纽约大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Department of Computer and Information Science, University of Macau(澳门大学计算机与信息科学系)
MindAligner: Explicit Brain Functional Alignment for Cross-Subject Visual Decoding from Limited fMRI Data
Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Nanjing University of Science and Technology(南京理工大学);The Chinese University of Hong Kong(香港中文大学);Tsinghua University(清华大学)
Beihang University(北京航空航天大学);Shenzhen Institute of Advanced Technology(深圳先进技术研究院);Xiaohongshu Inc(小红书公司);Zhongguancun Laboratory(中关村实验室);Nanchang Institute of Technology(南昌理工学院)
MinD-3D++: Advancing fMRI-Based 3D Reconstruction with High-Quality Textured Mesh Generation and a Comprehensive Dataset
Fudan University(复旦大学);Fudan ISTBI–ZJNU Algorithm Centre for Brain-Inspired Intelligence(复旦大学-浙江师范大学脑启发智能算法中心);Zhejiang Normal University(浙江师范大学);Shanghai Innovation Institute(上海创新研究院);ETH Zürich(苏黎世联邦理工学院);Northwestern Polytechnical University(西北工业大学)
Mind the Domain Gap: Measuring the Domain Gap Between Real-World and Synthetic Point Clouds for Automated Driving Development
Technical University of Munich(慕尼黑技术大学);Chair of Geoinformatics, Technical University of Munich(地理信息学教授职位,慕尼黑技术大学);Chair of Photogrammetry and Remote Sensing, Technical University of Munich(摄影测量与遥感教授职位,慕尼黑技术大学);Department of Geoinformatics, University of Applied Science Munich(慕尼黑应用科学大学地理信息学系)
Mind-to-Image: Projecting Visual Mental Imagination of the Brain from fMRI
Obvious Research(奥布维研究机构);GHU Paris Psychiatrie et Neurosciences(巴黎精神病学与神经科学大学医院集团);Université Paris Cité(巴黎西岱大学);Centre Imagerie du Nord(北方影像中心);Raidium(雷迪姆公司);FrontLab, Sorbonne Université(索邦大学前沿实验室);Institut du Cerveau(大脑研究所);Centre de NeuroImagerie de Recherche, Sorbonne Université(索邦大学神经影像研究中心);Institut des Systèmes Intelligents et de la Robotique, Sorbonne Université(索邦大学智能系统与机器人研究所)
Mind the Discriminability Trap in Source-Free Cross-domain Few-shot Learning
School of Computer Science and Technology, Huazhong University of Science and Technology(华中科技大学计算机科学与技术学院);National Key Laboratory for Multimedia Information Processing, Peking University(北京大学多媒体信息处理国家重点实验室)
Mind What Matters for Reasoning: Aligning Cross-Modal Attention via Selective Probability Mass Concentration
University of Technology Sydney(悉尼科技大学);East China Normal University(华东师范大学);The University of New South Wales(新南威尔士大学);Shanghai Jiaotong University(上海交通大学);The Education University of Hong Kong(香港教育大学)
The Mind's Eye: A Multi-Faceted Reward Framework for Guiding Visual Metaphor Generation
NICE Research Group(NICE研究组);Centre for Translation Studies(翻译研究中心);Institute for People-Centred AI(以人为本的人工智能研究所);School of Computer Science & Electronic Engineering, University of Surrey, UK(Surrey大学计算机科学与电子工程学院)
Active Learning for Automated Visual Inspection of Manufactured Products
University of Ljubljana(卢布尔雅那大学);Jožef Stefan International Postgraduate School(约热夫·斯特凡国际研究生院);Philips Consumer Lifestyle BV(飞利浦消费生活方式有限公司);Qlector d.o.o.(Qlector有限责任公司);Jožef Stefan Institute(约热夫·斯特凡研究所)