Five Points to Check when Comparing Visual Perception in Humans and Machines
University of Tübingen（图宾根大学）；Bernstein Center for Computational Neuroscience（伯恩斯坦计算神经科学中心）；Werner Reichardt Centre for Integrative Neuroscience（维尔纳·赖夏特整合神经科学中心）；Volkswagen Group Machine Learning Research Lab（大众集团机器学习研究实验室）
Christina M. Funke, Judy Borowski, Karolina Stosio, Wieland Brendel, Thomas S. A. Wallis, Matthias Bethge
备注：
V3: minor changes like in published JOV version (https://doi.org/10.1167/jov.21.3.16) V2: New title; added general section (checklist); manuscript restructured such that each case study is one chapter; adversarial examples in first study replaced by different analysis
With the rise of machines to human-level performance in complex recognition tasks, a growing amount of work is directed towards comparing information processing in humans and machines. These studies are an exciting chance to learn about one system by studying the other. Here, we propose ideas on how to design, conduct and interpret experiments such that they adequately support the investigation of mechanisms when comparing human and machine perception. We demonstrate and apply these ideas through three case studies. The first case study shows how human bias can affect how we interpret results, and that several analytic tools can help to overcome this human reference point. In the second case study, we highlight the difference between necessary and sufficient mechanisms in visual reasoning tasks. Thereby, we show that contrary to previous suggestions, feedback mechanisms might not be necessary for the tasks in question. The third case study highlights the importance of aligning experimental conditions. We find that a previously-observed difference in object recognition does not hold when adapting the experiment to make conditions more equitable between humans and machines. In presenting a checklist for comparative studies of visual reasoning in humans and machines, we hope to highlight how to overcome potential pitfalls in design or inference.
LiDAR sensors are used widely in Autonomous Vehicles for better perceiving the environment which enables safer driving decisions. Recent work has demonstrated serious LiDAR spoofing attacks with alarming consequences. In particular, model-level LiDAR spoofing attacks aim to inject fake depth measurements to elicit ghost objects that are erroneously detected by 3D Object Detectors, resulting in hazardous driving decisions. In this work, we explore the use of motion as a physical invariant of genuine objects for detecting such attacks. Based on this, we propose a general methodology, 3D Temporal…
Many surface cues support three-dimensional shape perception, but people can sometimes still see shape when these features are missing -- in extreme cases, even when an object is completely occluded, as when covered with a draped cloth. We propose a framework for 3D shape perception that explains perception in both typical and atypical cases as analysis-by-synthesis, or inference in a generative model of image formation: the model integrates intuitive physics to explain how shape can be inferred from deformations it causes to other objects, as in cloth-draping. Behavioral and computational stu…
Texture Representations in Deep Vision Models: Comparing CNNs, Vision Transformers, and Human Perception
Department of Neuroscience International School for Advanced Studies (SISSA) Trieste Italy（神经科学系国际先进研究学院（SISSA）特里埃斯特意大利）；Department of Language and Translation Sciences Pompeu Fabra University Barcelona Spain（语言与翻译科学系庞培法布拉大学巴塞罗那西班牙）；ICREA Barcelona Spain（ICREA巴塞罗那西班牙）；Department of Data Science International School for Advanced Studies (SISSA) Trieste Italy（数据科学系国际先进研究学院（SISSA）特里埃斯特意大利）
Brandeis University（布兰迪斯大学）；Mitsubishi Electric Research Laboratories（三菱电机研究实验室）
We consider the object recognition problem in autonomous driving using automotive radar sensors. Comparing to Lidar sensors, radar is cost-effective and robust in all-weather conditions for perception in autonomous driving. However, radar signals suffer from low angular resolution and precision in recognizing surrounding objects. To enhance the capacity of automotive radar, in this work, we exploit the temporal information from successive ego-centric bird-eye-view radar image frames for radar object recognition. We leverage the consistency of an object's existence and attributes (size, orienta…
Valeo R&D Egypt（法雷奥研发埃及）；Valeo DAR Germany（法雷奥DAR德国）；Technische Universität Ilmenau（伊尔梅瑙工业大学）；Valeo Ireland（法雷奥爱尔兰）
Deep neural networks (DNNs) have accomplished impressive success in various applications, including autonomous driving perception tasks, in recent years. On the other hand, current deep neural networks are easily fooled by adversarial attacks. This vulnerability raises significant concerns, particularly in safety-critical applications. As a result, research into attacking and defending DNNs has gained much coverage. In this work, detailed adversarial attacks are applied on a diverse multi-task visual perception deep network across distance estimation, semantic segmentation, motion detection, a…
Qingdao University（青岛大学）；New York University（纽约大学）；Shandong University（山东大学）；De Montfort University（德蒙福特大学）
With the wide applications of colored point cloud in many fields, point cloud perceptual quality assessment plays a vital role in the visual communication systems owing to the existence of quality degradations introduced in various stages. However, the existing point cloud quality assessments ignore the mechanism of human visual system (HVS) which has an important impact on the accuracy of the perceptual quality assessment. In this paper, a progressive knowledge transfer based on human visual perception mechanism for perceptual quality assessment of point clouds (PKT-PCQA) is proposed. The PKT…
For many of the physical phenomena around us, we have developed sophisticated models explaining their behavior. Nevertheless, inferring specifics from visual observations is challenging due to the high number of causally underlying physical parameters -- including material properties and external forces. This paper addresses the problem of inferring such latent physical properties from observations. Our solution is an iterative refinement procedure with simulation at its core. The algorithm gradually updates the physical model parameters by running a simulation of the observed phenomenon and c…
