首页/论文/2005.04153

arXiv 2005.041532020-05cs.NEcs.CVcs.LGstat.ML

A Hybrid Method for Training Convolutional Neural Networks

Universidade da Beira Interior（贝拉内格罗大学）；Instituto de Telecomunicações（电信研究所）

Vasco Lopes, Paulo Fazendeiro

备注：: 1 figure, 6 pages

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

英文摘要相关论文

英文摘要：

Artificial Intelligence algorithms have been steadily increasing in popularity and usage. Deep Learning, allows neural networks to be trained using huge datasets and also removes the need for human extracted features, as it automates the feature learning process. In the hearth of training deep neural networks, such as Convolutional Neural Networks, we find backpropagation, that by computing the gradient of the loss function with respect to the weights of the network for a given input, it allows the weights of the network to be adjusted to better perform in the given task. In this paper, we propose a hybrid method that uses both backpropagation and evolutionary strategies to train Convolutional Neural Networks, where the evolutionary strategies are used to help to avoid local minimas and fine-tune the weights, so that the network achieves higher accuracy results. We show that the proposed hybrid method is capable of improving upon regular training in the task of image classification in CIFAR-10, where a VGG16 model was used and the final test results increased 0.61%, in average, when compared to using only backpropagation.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

DeepBrain: Functional Representation of Neural In-Situ Hybridization Images for Gene Ontology Classification Using Deep Convolutional Autoencoders

Bar-Ilan University（巴伊兰大学）；University of Maryland（马里兰大学）

This paper presents a novel deep learning-based method for learning a functional representation of mammalian neural images. The method uses a deep convolutional denoising autoencoder (CDAE) for generating an invariant, compact representation of in situ hybridization (ISH) images. While most existing methods for bio-imaging analysis were not developed to handle images with highly complex anatomical structures, the results presented in this paper show that functional representation extracted by CDAE can help learn features of functional gene ontology categories for their classification in a high…

2017-11-27cs.CVcs.LGcs.NE

Method for Hybrid Precision Convolutional Neural Network Representation

This invention addresses fixed-point representations of convolutional neural networks (CNN) in integrated circuits. When quantizing a CNN for a practical implementation there is a trade-off between the precision used for operations between coefficients and data and the accuracy of the system. A homogenous representation may not be sufficient to achieve the best level of performance at a reasonable cost in implementation complexity or power consumption. Parsimonious ways of representing data and coefficients are needed to improve power efficiency and throughput while maintaining accuracy of a C…

2018-07-24cs.NE

Recent Advances in Convolutional Neural Networks

Nanyang Technological University（南洋理工大学）

In the last few years, deep learning has led to very good performance on a variety of problems, such as visual recognition, speech recognition and natural language processing. Among different types of deep neural networks, convolutional neural networks have been most extensively studied. Leveraging on the rapid growth in the amount of the annotated data and the great improvements in the strengths of graphics processor units, the research on convolutional neural networks has been emerged swiftly and achieved state-of-the-art results on various tasks. In this paper, we provide a broad survey of …

2017-10-19cs.CVcs.LGcs.NE

Quaternion-Valued Convolutional Neural Network Applied for Acute Lymphoblastic Leukemia Diagnosis

Escuela Superior Politécnica del Litoral（滨海理工学院）；Universidade Estadual de Campinas（坎皮纳斯州立大学）

The field of neural networks has seen significant advances in recent years with the development of deep and convolutional neural networks. Although many of the current works address real-valued models, recent studies reveal that neural networks with hypercomplex-valued parameters can better capture, generalize, and represent the complexity of multidimensional data. This paper explores the quaternion-valued convolutional neural network application for a pattern recognition task from medicine, namely, the diagnosis of acute lymphoblastic leukemia. Precisely, we compare the performance of real-va…

2021-12-13cs.CVcs.LGcs.NE

Implementation of Training Convolutional Neural Networks

University of Chinese Academy of Sciences（中国科学院大学）

Deep learning refers to the shining branch of machine learning that is based on learning levels of representations. Convolutional Neural Networks (CNN) is one kind of deep neural network. It can study concurrently. In this article, we gave a detailed analysis of the process of CNN algorithm both the forward process and back propagation. Then we applied the particular convolutional neural network to implement the typical face recognition problem by java. Then, a parallel strategy was proposed in section4. In addition, by measuring the actual time of forward and backward computing, we analysed t…

2015-06-04cs.CVcs.LGcs.NE

One weird trick for parallelizing convolutional neural networks

Google Inc.（谷歌公司）

I present a new way to parallelize the training of convolutional neural networks across multiple GPUs. The method scales significantly better than all alternatives when applied to modern convolutional neural networks.

2014-04-26cs.DCcs.LGcs.NE

更多相关论文收起更多论文

Convolutional Neural Network(CNN/ConvNet) in Stock Price Movement Prediction

Shiv Nadar University（希夫·纳达尔大学）

With technological advancements and the exponential growth of data, we have been unfolding different capabilities of neural networks in different sectors. In this paper, I have tried to use a specific type of Neural Network known as Convolutional Neural Network(CNN/ConvNet) in the stock market. In other words, I have tried to construct and train a convolutional neural network on past stock prices data and then tried to predict the movement of stock price i.e. whether the stock price would rise or fall, in the coming time.

2021-06-03cs.AIcs.CVcs.LG

A Hybrid Differential Evolution Approach to Designing Deep Convolutional Neural Networks for Image Classification

Victoria University of Wellington（惠灵顿维多利亚大学）

Convolutional Neural Networks (CNNs) have demonstrated their superiority in image classification, and evolutionary computation (EC) methods have recently been surging to automatically design the architectures of CNNs to save the tedious work of manually designing CNNs. In this paper, a new hybrid differential evolution (DE) algorithm with a newly added crossover operator is proposed to evolve the architectures of CNNs of any lengths, which is named DECNN. There are three new ideas in the proposed DECNN method. Firstly, an existing effective encoding scheme is refined to cater for variable-leng…

2018-08-22cs.AIcs.CVcs.LG

Learning Convolutional Neural Networks using Hybrid Orthogonal Projection and Estimation

York University（约克大学）

Convolutional neural networks (CNNs) have yielded the excellent performance in a variety of computer vision tasks, where CNNs typically adopt a similar structure consisting of convolution layers, pooling layers and fully connected layers. In this paper, we propose to apply a novel method, namely Hybrid Orthogonal Projection and Estimation (HOPE), to CNNs in order to introduce orthogonality into the CNN structure. The HOPE model can be viewed as a hybrid model to combine feature extraction using orthogonal linear projection with mixture models. It is an effective model to extract useful informa…

2016-09-11cs.CVcs.NE

Coarse and fine-grained automatic cropping deep convolutional neural network

School of Computer Science and Information Engineering, Hefei University of Technology（合肥工业大学计算机与信息工程学院）

The existing convolutional neural network pruning algorithms can be divided into two categories: coarse-grained clipping and fine-grained clipping. This paper proposes a coarse and fine-grained automatic pruning algorithm, which can achieve more efficient and accurate compression acceleration for convolutional neural networks. First, cluster the intermediate feature maps of the convolutional neural network to obtain the network structure after coarse-grained clipping, and then use the particle swarm optimization algorithm to iteratively search and optimize the structure. Finally, the optimal n…

2020-10-14cs.CVcs.NE

Fast Algorithms for Convolutional Neural Networks

Nervana Systems（Nervana系统公司）

Deep convolutional neural networks take GPU days of compute time to train on large data sets. Pedestrian detection for self driving cars requires very low latency. Image recognition for mobile phones is constrained by limited processing resources. The success of convolutional neural networks in these situations is limited by how fast we can compute them. Conventional FFT based convolution is fast for large filters, but state of the art convolutional neural networks use small, 3x3 filters. We introduce a new class of fast algorithms for convolutional neural networks using Winograd's minimal fil…

2015-11-10cs.LGcs.NE

Generative Models for Stochastic Processes Using Convolutional Neural Networks

The present paper aims to demonstrate the usage of Convolutional Neural Networks as a generative model for stochastic processes, enabling researchers from a wide range of fields (such as quantitative finance and physics) to develop a general tool for forecasts and simulations without the need to identify/assume a specific system structure or estimate its parameters.

2018-01-09cs.NEphysics.comp-phq-fin.CP

Spectral classification using convolutional neural networks

Masaryk University（马萨里克大学）

There is a great need for accurate and autonomous spectral classification methods in astrophysics. This thesis is about training a convolutional neural network (ConvNet) to recognize an object class (quasar, star or galaxy) from one-dimension spectra only. Author developed several scripts and C programs for datasets preparation, preprocessing and postprocessing of the data. EBLearn library (developed by Pierre Sermanet and Yann LeCun) was used to create ConvNets. Application on dataset of more than 60000 spectra yielded success rate of nearly 95%. This thesis conclusively proved great potentia…

2014-12-29astro-ph.IMcs.CVcs.NE

用于图像分类的卷积脉冲神经网络

Convolutional Spiking Neural Network for Image Classification

提出一种在 CoLaNET 脉冲神经网络中加入固定卷积层的图像分类方法，并利用同域代表性图像确定卷积权重，且在 NEOVISION2 基准上验证。

2025-05-13cs.NE

Non-uniqueness phenomenon of object representation in modelling IT cortex by deep convolutional neural network (DCNN)

Recently DCNN (Deep Convolutional Neural Network) has been advocated as a general and promising modelling approach for neural object representation in primate inferotemporal cortex. In this work, we show that some inherent non-uniqueness problem exists in the DCNN-based modelling of image object representations. This non-uniqueness phenomenon reveals to some extent the theoretical limitation of this general modelling approach, and invites due attention to be taken in practice.

2019-06-06cs.NEq-bio.NC

Combining Neuro-Evolution of Augmenting Topologies with Convolutional Neural Networks

Current deep convolutional networks are fixed in their topology. We explore the possibilites of making the convolutional topology a parameter itself by combining NeuroEvolution of Augmenting Topologies (NEAT) with Convolutional Neural Networks (CNNs) and propose such a system using blocks of Residual Networks (ResNets). We then explain how our suggested system can only be built once additional optimizations have been made, as genetic algorithms are way more demanding than training per backpropagation. On the way there we explain most of those buzzwords and offer a gentle and brief introduction…

2022-10-20cs.AIcs.NE