Most of the Zero-Shot Learning (ZSL) algorithms currently use pre-trained models as their feature extractors, which are usually trained on the ImageNet data set by using deep neural networks. The richness of the feature information embedded in the pre-trained models can help the ZSL model extract more useful features from its limited training samples. However, sometimes the difference between the training data set of the current ZSL task and the ImageNet data set is too large, which may lead to the use of pre-trained models has no obvious help or even negative impact on the performance of the ZSL model. To solve this problem, this paper proposes a biologically inspired feature enhancement framework for ZSL. Specifically, we design a dual-channel learning framework that uses auxiliary data sets to enhance the feature extractor of the ZSL model and propose a novel method to guide the selection of the auxiliary data sets based on the knowledge of biological taxonomy. Extensive experimental results show that our proposed method can effectively improve the generalization ability of the ZSL model and achieve state-of-the-art results on three benchmark ZSL tasks. We also explained the experimental phenomena through the way of feature visualization.
Harvard University（哈佛大学）；Massachusetts Institute of Technology（麻省理工学院）
A convolutional neural network strongly robust to adversarial perturbations at reasonable computational and performance cost has not yet been demonstrated. The primate visual ventral stream seems to be robust to small perturbations in visual stimuli but the underlying mechanisms that give rise to this robust perception are not understood. In this work, we investigate the role of two biologically plausible mechanisms in adversarial robustness. We demonstrate that the non-uniform sampling performed by the primate retina and the presence of multiple receptive fields with a range of receptive fiel…
Starmind AG（Starmind公司）；University of Zurich（苏黎世大学）
This work proposes a model for continual learning on tasks involving temporal sequences, specifically, human motions. It improves on a recently proposed brain-inspired replay model (BI-R) by building a biologically-inspired conditional temporal variational autoencoder (BI-CTVAE), which instantiates a latent mixture-of-Gaussians for class representation. We investigate a novel continual-learning-to-generate (CL2Gen) scenario where the model generates motion sequences of different classes. The generative accuracy of the model is tested over a set of tasks. The final classification accuracy of BI…
The ability to look multiple times through a series of pose-adjusted glimpses is fundamental to human vision. This critical faculty allows us to understand highly complex visual scenes. Short term memory plays an integral role in aggregating the information obtained from these glimpses and informing our interpretation of the scene. Computational models have attempted to address glimpsing and visual attention but have failed to incorporate the notion of memory. We introduce a novel, biologically inspired visual working memory architecture that we term the Hebb-Rosenblatt memory. We subsequently…
South China University of Technology（华南理工大学）；University of Macau（澳门大学）；University of Technology Sydney（悉尼科技大学）
Visual tracking is challenging due to image variations caused by various factors, such as object deformation, scale change, illumination change and occlusion. Given the superior tracking performance of human visual system (HVS), an ideal design of biologically inspired model is expected to improve computer visual tracking. This is however a difficult task due to the incomplete understanding of neurons' working mechanism in HVS. This paper aims to address this challenge based on the analysis of visual cognitive mechanism of the ventral stream in the visual cortex, which simulates shallow neuron…
Lateral connections play an important role for sensory processing in visual cortex by supporting discriminable neuronal responses even to highly similar features. In the present work, we show that establishing a biologically inspired Mexican hat lateral connectivity profile along the filter domain can significantly improve the classification accuracy of a variety of lightweight convolutional neural networks without the addition of trainable network parameters. Moreover, we demonstrate that it is possible to analytically determine the stationary distribution of modulated filter activations and …
Feature extraction and matching are among central problems of computer vision. It is inefficent to search features over all locations and scales. Neurophysiological evidence shows that to locate objects in a digital image the human visual system employs visual attention to a specific object while ignoring others. The brain also has a mechanism to search from coarse to fine. In this paper, we present a feature extractor and an associated hierarchical searching model to simulate such processes. With the hierarchical representation of the object, coarse scanning is done through the matching of th…
Centre for Research and Technology Hellas（希腊研究与技术中心）
In this work we introduce a biologically inspired long-range skip connection for the UNet architecture that relies on the perceptual illusion of hybrid images, being images that simultaneously encode two images. The fusion of early encoder features with deeper decoder ones allows UNet models to produce finer-grained dense predictions. While proven in segmentation tasks, the network's benefits are down-weighted for dense regression tasks as these long-range skip connections additionally result in texture transfer artifacts. Specifically for depth estimation, this hurts smoothness and introduces…
We present a novel cortically-inspired image completion algorithm. It uses a five dimensional sub-Riemannian cortical geometry modelling the orientation, spatial frequency and phase selective behavior of the cells in the visual cortex. The algorithm extracts the orientation, frequency and phase information existing in a given two dimensional corrupted input image via a Gabor transform and represent those values in terms of cortical cell output responses in the model geometry. Then it performs completion via a diffusion concentrated in a neighbourhood along the neural connections within the mod…
Within the domain of texture classification, a lot of effort has been spent on local descriptors, leading to many powerful algorithms. However, preprocessing techniques have received much less attention despite their important potential for improving the overall classification performance. We address this question by proposing a novel, simple, yet very powerful biologically-inspired filtering (BF) which simulates the performance of human retina. In the proposed approach, given a texture image, after applying a DoG filter to detect the "edges", we first split the filtered image into two "maps" …
Center for Neural Science（神经科学中心）；New York University（纽约大学）；Courant Inst. of Mathematical Sciences（库朗数学科学研究所）；Howard Hughes Medical Institute（霍华德·休斯医学研究所）
We develop a model for representing visual texture in a low-dimensional feature space, along with a novel self-supervised learning objective that is used to train it on an unlabeled database of texture images. Inspired by the architecture of primate visual cortex, the model uses a first stage of oriented linear filters (corresponding to cortical area V1), consisting of both rectified units (simple cells) and pooled phase-invariant units (complex cells). These responses are processed by a second stage (analogous to cortical area V2) consisting of convolutional filters followed by half-wave rect…
Computational neuroscience studies that have examined human visual system through functional magnetic resonance imaging (fMRI) have identified a model where the mammalian brain pursues two distinct pathways (for recognition of biological movement tasks). In the brain, dorsal stream analyzes the information of motion (optical flow), which is the fast features, and ventral stream (form pathway) analyzes form information (through active basis model based incremental slow feature analysis ) as slow features. The proposed approach suggests the motion perception of the human visual system composes o…
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；The State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所复杂系统管理与控制国家重点实验室）；CAS Center for Excellence in Brain Science and Intelligence Technology（中国科学院脑科学与智能技术卓越创新中心）；Nanchang University（南昌大学）
Biologically inspired model (BIM) for image recognition is a robust computational architecture, which has attracted widespread attention. BIM can be described as a four-layer structure based on the mechanisms of the visual cortex. Although the performance of BIM for image recognition is robust, it takes the randomly selected ways for the patch selection, which is sightless, and results in heavy computing burden. To address this issue, we propose a novel patch selection method with oriented Gaussian-Hermite moment (PSGHM), and we enhanced the BIM based on the proposed PSGHM, named as PBIM. In c…
Peking University（北京大学）；Cooperative Medianet Innovation Center（合作媒体网络创新中心）；Beijing Institute of Technology（北京理工大学）
Despite a lot of research efforts devoted in recent years, how to efficiently learn long-term dependencies from sequences still remains a pretty challenging task. As one of the key models for sequence learning, recurrent neural network (RNN) and its variants such as long short term memory (LSTM) and gated recurrent unit (GRU) are still not powerful enough in practice. One possible reason is that they have only feedforward connections, which is different from the biological neural system that is typically composed of both feedforward and feedback connections. To address this problem, this paper…
Università di Bologna（博洛尼亚大学）；CAMS Center of Mathematical Analysis, CNRS-EHESS（CAMS数学分析中心，法国国家科学研究中心-社会科学高等研究院）
The state of the art in many computer vision tasks is represented by Convolutional Neural Networks (CNNs). Although their hierarchical organization and local feature extraction are inspired by the structure of primate visual systems, the lack of lateral connections in such architectures critically distinguishes their analysis from biological object processing. The idea of enriching CNNs with recurrent lateral connections of convolutional type has been put into practice in recent years, in the form of learned recurrent kernels with no geometrical constraints. In the present work, we introduce b…
Biologically Inspired Deep Learning Approaches for Fetal Ultrasound Image Classification
Skolkovo Institute of Science and Technology（斯克尔科沃科学与技术研究所）；BIMAI-Lab, Biomedically Informed Artificial Intelligence Laboratory, University of Sharjah（沙迦大学BIMAI实验室，生物医学引导人工智能实验室）
CNRS（法国国家科学研究中心）；UNIC（神经信息与复杂性实验室）；Ceremade（Ceremade研究中心）；Univ. Paris-Dauphine（巴黎第九大学）；Institut de Neurosciences de la Timone（蒂莫内神经科学研究所）；UMR 7289 CNRS/Aix-Marseille Université（法国国家科学研究中心/艾克斯-马赛大学 UMR 7289 联合研究单位）
Perception is often described as a predictive process based on an optimal inference with respect to a generative model. We study here the principled construction of a generative model specifically crafted to probe motion perception. In that context, we first provide an axiomatic, biologically-driven derivation of the model. This model synthesizes random dynamic textures which are defined by stationary Gaussian distributions obtained by the random aggregation of warped patterns. Importantly, we show that this model can equivalently be described as a stochastic partial differential equation. Usi…
By and large, existing computational models of visual attention tacitly assume perfect vision and full access to the stimulus and thereby deviate from foveated biological vision. Moreover, modeling top-down attention is generally reduced to the integration of semantic features without incorporating the signal of a high-level visual tasks that have been shown to partially guide human attention. We propose the Neural Visual Attention (NeVA) algorithm to generate visual scanpaths in a top-down manner. With our method, we explore the ability of neural networks on which we impose a biologically-ins…
CL-BioGAN: Biologically-Inspired Cross-Domain Continual Learning for Hyperspectral Anomaly Detection
Key Laboratory of Collaborative Intelligence Systems, Ministry of Education, School of Computer Science and Technology, Xidian University（协同智能系统重点实验室、教育部、计算机科学与技术学院、西安电子科技大学）；Key Laboratory of Intelligent Perception and Image Understanding of Ministry of Education of China, School of Artificial Intelligence, Xidian University（智能感知与图像理解重点实验室、教育部、人工智能学院、西安电子科技大学）；Key Laboratory of Collaborative Intelligence Systems, Ministry of Education, Xidian University（协同智能系统重点实验室、教育部、西安电子科技大学）
DualGazeNet: A Biologically Inspired Dual-Gaze Query Network for Salient Object Detection
School of Computation, Information and Technology, Technical University of Munich（计算信息与技术学院，慕尼黑技术大学）；Department of Informatics, Technical University of Munich（信息学院，慕尼黑技术大学）；University Research and Innovation Center, Obuda University（奥布达大学研究与创新中心）；State Key Laboratory for Novel Software Technology, Nanjing University Suzhou Campus（新型软件技术国家重点实验室，南京大学苏州校区）；School of Science and Technology, Nanjing University Suzhou Campus（科学与技术学院，南京大学苏州校区）；Department of Computer Science and Technology, Tsinghua University（计算机科学与技术系，清华大学）
Pooling is a ubiquitous operation in image processing algorithms that allows for higher-level processes to collect relevant low-level features from a region of interest. Currently, max-pooling is one of the most commonly used operators in the computational literature. However, it can lack robustness to outliers due to the fact that it relies merely on the peak of a function. Pooling mechanisms are also present in the primate visual cortex where neurons of higher cortical areas pool signals from lower ones. The receptive fields of these neurons have been shown to vary according to the contrast …
Attention via Synaptic Plasticity is All You Need: A Biologically Inspired Spiking Neuromorphic Transformer
Department of Electronics and Communication Engineering, National Institute of Technology Allahabad, Prayagraj, 211004, India（国家理工学院阿姆利达分校电子与通信工程系）；Centre for Nanotechnology, Indian Institute of Technology Roorkee, Roorkee, Uttarakhand, 247667, India（印度理工学院罗奥基分校纳米技术中心）
Modern recognition systems require large amounts of supervision to achieve accuracy. Adapting to new domains requires significant data from experts, which is onerous and can become too expensive. Zero-shot learning requires an annotated set of attributes for a novel category. Annotating the full set of attributes for a novel category proves to be a tedious and expensive task in deployment. This is especially the case when the recognition domain is an expert domain. We introduce a new field-guide-inspired approach to zero-shot annotation where the learner model interactively asks for the most u…
UiT The Arctic University of Norway（特罗姆瑟大学（挪威北极圈大学））
Recent work has shown that label-efficient few-shot learning through self-supervision can achieve promising medical image segmentation results. However, few-shot segmentation models typically rely on prototype representations of the semantic classes, resulting in a loss of local information that can degrade performance. This is particularly problematic for the typically large and highly heterogeneous background class in medical image segmentation problems. Previous works have attempted to address this issue by learning additional prototypes for each class, but since the prototypes are based on…
