Deep neural networks (DNNs) are machine learning algorithms that have revolutionised computer vision due to their remarkable successes in tasks like object classification and segmentation. The success of DNNs as computer vision algorithms has led to the suggestion that DNNs may also be good models of human visual perception. We here review evidence regarding current DNNs as adequate behavioural models of human core object recognition. To this end, we argue that it is important to distinguish between statistical tools and computational models, and to understand model quality as a multidimensional concept where clarity about modelling goals is key. Reviewing a large number of psychophysical and computational explorations of core object recognition performance in humans and DNNs, we argue that DNNs are highly valuable scientific tools but that as of today DNNs should only be regarded as promising -- but not yet adequate -- computational models of human core object recognition behaviour. On the way we dispel a number of myths surrounding DNNs in vision science.
Korea Advanced Institute of Science and Technology（韩国科学技术院）；Okinawa Institute of Science and Technology（冲绳科学技术大学院大学）
This study investigates how adequate coordination among the different cognitive processes of a humanoid robot can be developed through end-to-end learning of direct perception of visuomotor stream. We propose a deep dynamic neural network model built on a dynamic vision network, a motor generation network, and a higher-level network. The proposed model was designed to process and to integrate direct perception of dynamic visuomotor patterns in a hierarchical model characterized by different spatial and temporal constraints imposed on each level. We conducted synthetic robotic experiments in wh…
Donders Institute for Brain, Cognition and Behaviour（唐德斯大脑、认知与行为研究所）；PAL robotics（PAL机器人公司）；Technical University of Munich（慕尼黑工业大学）
Self/other distinction and self-recognition are important skills for interacting with the world, as it allows humans to differentiate own actions from others and be self-aware. However, only a selected group of animals, mainly high order mammals such as humans, has passed the mirror test, a behavioural experiment proposed to assess self-recognition abilities. In this paper, we describe self-recognition as a process that is built on top of body perception unconscious mechanisms. We present an algorithm that enables a robot to perform non-appearance self-recognition on a mirror and distinguish i…
Many surface cues support three-dimensional shape perception, but people can sometimes still see shape when these features are missing -- in extreme cases, even when an object is completely occluded, as when covered with a draped cloth. We propose a framework for 3D shape perception that explains perception in both typical and atypical cases as analysis-by-synthesis, or inference in a generative model of image formation: the model integrates intuitive physics to explain how shape can be inferred from deformations it causes to other objects, as in cloth-draping. Behavioral and computational stu…
I Spy With My Model's Eye: Visual Search as a Behavioural Test for MLLMs
Leverhulme Centre for the Future of Intelligence, University of Cambridge（未来智能研究中心、剑桥大学）；Department of Engineering, University of Cambridge（工程系、剑桥大学）；Department of Psychology, University of Cambridge（心理学系、剑桥大学）；Department of Computer Science, University of Cambridge（计算机科学系、剑桥大学）
The visual world is very rich and generally too complex to perceive in its entirety. Yet only certain features are typically required to adequately perform some task in a given situation. Rather than hardwire-in decisions about when and what to sense, this paper describes a robotic system whose behavioral policy can be set by verbal instructions it receives. These capabilities are demonstrated in an associated video showing the fully implemented system guiding the perception of a physical robot in simple scenario. The structure and functioning of the underlying natural language based symbolic …
Gustave Eiffel University（古斯塔夫·埃菲尔大学）；Cardiff University（卡迪夫大学）；Xi'an Jiaotong University（西安交通大学）；Zhongkai University of Agriculture and Engineering（仲恺农业工程学院）；University of Bath（巴斯大学）；Xidian University（西安电子科技大学）
Research on image quality assessment (IQA) remains limited mainly due to our incomplete knowledge about human visual perception. Existing IQA algorithms have been designed or trained with insufficient subjective data with a small degree of stimulus variability. This has led to challenges for those algorithms to handle complexity and diversity of real-world digital content. Perceptual evidence from human subjects serves as a grounding for the development of advanced IQA algorithms. It is thus critical to acquire reliable subjective data with controlled perception experiments that faithfully ref…
Self-supervised Adaptive Weighting for Cooperative Perception in V2V Communications
University of Warwick（华威大学）；University of Technology Sydney（悉尼科技大学）；University of Durham（杜伦大学）；Oracle for Research（甲骨文研究部门）；Southern University of Science and Technology（南方科技大学）；University of Reading（雷丁大学）
University of Tübingen（图宾根大学）；Bernstein Center for Computational Neuroscience（伯恩斯坦计算神经科学中心）；Werner Reichardt Centre for Integrative Neuroscience（维尔纳·赖夏特整合神经科学中心）；Volkswagen Group Machine Learning Research Lab（大众集团机器学习研究实验室）
With the rise of machines to human-level performance in complex recognition tasks, a growing amount of work is directed towards comparing information processing in humans and machines. These studies are an exciting chance to learn about one system by studying the other. Here, we propose ideas on how to design, conduct and interpret experiments such that they adequately support the investigation of mechanisms when comparing human and machine perception. We demonstrate and apply these ideas through three case studies. The first case study shows how human bias can affect how we interpret results,…
University of Tübingen（图宾根大学）；International Max Planck Research School for Intelligent Systems（马克斯·普朗克智能系统国际研究学院）
A few years ago, the first CNN surpassed human performance on ImageNet. However, it soon became clear that machines lack robustness on more challenging test cases, a major obstacle towards deploying machines "in the wild" and towards obtaining better computational models of human visual perception. Here we ask: Are we making progress in closing the gap between human and machine vision? To answer this question, we tested human observers on a broad range of out-of-distribution (OOD) datasets, recording 85,120 psychophysical trials across 90 participants. We then investigated a range of promising…
University of Texas at El Paso（德克萨斯大学埃尔帕索分校）；Cleveland State University（克利夫兰州立大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）
Computer vision applications in intelligent transportation systems (ITS) and autonomous driving (AD) have gravitated towards deep neural network architectures in recent years. While performance seems to be improving on benchmark datasets, many real-world challenges are yet to be adequately considered in research. This paper conducted an extensive literature review on the applications of computer vision in ITS and AD, and discusses challenges related to data, models, and complex urban environments. The data challenges are associated with the collection and labeling of training data and its rele…
University of Bristol（布里斯托大学）；King’s College London（伦敦国王学院）
We propose a computational model of visual search that incorporates Bayesian interpretations of the neural mechanisms that underlie categorical perception and saccade planning. To enable meaningful comparisons between simulated and human behaviours, we employ a gaze-contingent paradigm that required participants to classify occluded MNIST digits through a window that followed their gaze. The conditional independencies imposed by a separation of time scales in this task are embodied by constraints on the hierarchical structure of our model; planning and decision making are cast as a partially o…
Over the last decade, robotic perception algorithms have significantly benefited from the rapid advances in deep learning (DL). Indeed, a significant amount of the autonomy stack of different commercial and research platforms relies on DL for situational awareness, especially vision sensors. This work explores the potential of general-purpose DL perception algorithms, specifically detection and segmentation neural networks, for processing image-like outputs of advanced lidar sensors. Rather than processing the three-dimensional point cloud data, this is, to the best of our knowledge, the first…
University of California, Berkeley（加州大学伯克利分校）；Ford Greenfield Labs（福特格林菲尔德实验室）
Image-based learning methods for autonomous vehicle perception tasks require large quantities of labelled, real data in order to properly train without overfitting, which can often be incredibly costly. While leveraging the power of simulated data can potentially aid in mitigating these costs, networks trained in the simulation domain usually fail to perform adequately when applied to images in the real domain. Recent advances in domain adaptation have indicated that a shared latent space assumption can help to bridge the gap between the simulation and real domains, allowing the transference o…
Persona-aware and Explainable Bikeability Assessment: A Vision-Language Model Approach
University of Alabama（阿拉巴马大学）；University of Maryland, College Park（马里兰大学学院公园分校）；Stony Brook University（石溪大学）；University of Florida（佛罗里达大学）；Johns Hopkins University（约翰霍普金斯大学）
National Institute of Technology, Durgapur（印度杜尔加普尔国家理工学院）
Facial expressions convey non-verbal cues, which play an important role in interpersonal relations. Automatic recognition of human face based on facial expression can be an important component of natural human-machine interface. It may also be used in behavioural science. Although human can recognize the face practically without any effort, but reliable face recognition by machine is a challenge. This paper presents a new approach for recognizing the face of a person considering the expressions of the same human face at different instances of time. This methodology is developed combining Eigen…
Texas A&M University（得克萨斯农工大学）；University of Pennsylvania（宾夕法尼亚大学）；University of Texas at Austin（得克萨斯大学奥斯汀分校）
Decentralized coordination of a robot swarm requires addressing the tension between local perceptions and actions, and the accomplishment of a global objective. In this work, we propose to learn decentralized controllers based on solely raw visual inputs. For the first time, that integrates the learning of two key components: communication and visual perception, in one end-to-end framework. More specifically, we consider that each robot has access to a visual perception of the immediate surroundings, and communication capabilities to transmit and receive messages from other neighboring robots.…
Aristotle University of Thessaloniki（塞萨洛尼基亚里士多德大学）
The performance of supervised deep learning algorithms depends significantly on the scale, quality and diversity of the data used for their training. Collecting and manually annotating large amount of data can be both time-consuming and costly tasks to perform. In the case of tasks related to visual human-centric perception, the collection and distribution of such data may also face restrictions due to legislation regarding privacy. In addition, the design and testing of complex systems, e.g., robots, which often employ deep learning-based perception models, may face severe difficulties as eve…
University of Oxford（牛津大学）；Element AI（元素人工智能）；Polytechnique Montréal（蒙特利尔工程学院）；Mila（米拉魁北克人工智能研究所）
As deep reinforcement learning driven by visual perception becomes more widely used there is a growing need to better understand and probe the learned agents. Understanding the decision making process and its relationship to visual inputs can be very valuable to identify problems in learned behavior. However, this topic has been relatively under-explored in the research community. In this work we present a method for synthesizing visual inputs of interest for a trained agent. Such inputs or states could be situations in which specific actions are necessary. Further, critical states in which a …
BPCLIP: A Bottom-up Image Quality Assessment from Distortion to Semantics Based on CLIP
Harbin Institute of Technology（哈尔滨工业大学）；Nanjing University of Information Science and Technology（南京信息工程大学）；Sichuan University of Science & Engineering（四川工程职业技术学院）
Visual perception and language understanding are - fundamental components of human intelligence, enabling them to understand and reason about objects and their interactions. It is crucial for machines to have this capacity to reason using these two modalities to invent new robot-human collaborative systems. Recent advances in deep learning have built separate sophisticated representations of both visual scenes and languages. However, understanding the associations between the two modalities in a shared context for multimodal reasoning remains a challenge. Focusing on language and vision modali…
