Current research in convolutional neural networks (CNN) focuses mainly on changing the architecture of the networks, optimizing the hyper-parameters and improving the gradient descent. However, most work use only 3 standard families of operations inside the CNN, the convolution, the activation function, and the pooling. In this work, we propose a new family of operations based on the Green's function of the Laplacian, which allows the network to solve the Laplacian, to integrate any vector field and to regularize the field by forcing it to be conservative. Hence, the Green's function (GF) is the first operation that regularizes the 2D or 3D feature space by forcing it to be conservative and physically interpretable, instead of regularizing the norm of the weights. Our results show that such regularization allows the network to learn faster, to have smoother training curves and to better generalize, without any additional parameter. The current manuscript presents early results, more work is required to benchmark the proposed method.
Oxford Robotics Institute（牛津机器人研究所）；University of Oxford（牛津大学）
This paper proposes a computationally efficient approach to detecting objects natively in 3D point clouds using convolutional neural networks (CNNs). In particular, this is achieved by leveraging a feature-centric voting scheme to implement novel convolutional layers which explicitly exploit the sparsity encountered in the input. To this end, we examine the trade-off between accuracy and speed for different architectures and additionally propose to use an L1 penalty on the filter activations to further encourage sparsity in the intermediate representations. To the best of our knowledge, this i…
University of Edinburgh（爱丁堡大学）；The Alan Turing Institute（艾伦·图灵研究所）
Neural networks pose a privacy risk to training data due to their propensity to memorise and leak information. Focusing on image classification, we show that neural networks also unintentionally memorise unique features even when they occur only once in training data. An example of a unique feature is a person's name that is accidentally present on a training image. Assuming access to the inputs and outputs of a trained model, the domain of the training data, and knowledge of unique features, we develop a score estimating the model's sensitivity to a unique feature by comparing the KL divergen…
Deep unrolling for learning optimal spatially varying regularisation parameters for Total Generalised Variation
University of Cambridge（剑桥大学）；School of Mathematical Sciences, Queen Mary University of London（伦敦玛丽女王大学数学科学学院）；Physikalisch-Technische Bundesanstalt (PTB)（德国联邦物理技术研究院）
University College London（伦敦大学学院）；University College London Hospitals（伦敦大学学院医院）；Wellcome/EPSRC Centre for Surgical and Interventional Science, UCL（惠康/EPSRC 外科与介入科学中心，伦敦大学学院）
Brain tumour segmentation plays a key role in computer-assisted surgery. Deep neural networks have increased the accuracy of automatic segmentation significantly, however these models tend to generalise poorly to different imaging modalities than those for which they have been designed, thereby limiting their applications. For example, a network architecture initially designed for brain parcellation of monomodal T1 MRI can not be easily translated into an efficient tumour segmentation network that jointly utilises T1, T1c, Flair and T2 MRI. To tackle this, we propose a novel scalable multimoda…
Convolutional Neural Networks (CNNs) are the state-of-the-art algorithms for the processing of images. However the configuration and training of these networks is a complex task requiring deep domain knowledge, experience and much trial and error. Using genetic algorithms, competitive CNN topologies for image recognition can be produced for any specific purpose, however in previous work this has come at high computational cost. In this work two novel approaches are presented to the utilisation of these algorithms, effective in reducing complexity and training time by nearly 20%. This is accomp…
We propose a robust variant of boosting forest to the various adversarial defense methods, and apply it to enhance the robustness of the deep neural network. We retain the deep network architecture, weights, and middle layer features, then install gradient boosting forest to select the features from each layer of the deep network, and predict the target. For training each decision tree, we propose a novel conservative and greedy trade-off, with consideration for less misprediction instead of pure gain functions, therefore being suboptimal and conservative. We actively increase tree depth to re…
Data Science Institute（数据科学研究所）；Imperial College London（伦敦帝国理工学院）
Deep learning using multi-layer neural networks (NNs) architecture manifests superb power in modern machine learning systems. The trained Deep Neural Networks (DNNs) are typically large. The question we would like to address is whether it is possible to simplify the NN during training process to achieve a reasonable performance within an acceptable computational time. We presented a novel approach of optimising a deep neural network through regularisation of net- work architecture. We proposed regularisers which support a simple mechanism of dropping neurons during a network training process. …
The University of British Columbia（不列颠哥伦比亚大学）；Xtract AI；Computational Geosciences Inc.（计算地球科学公司）
Convolutional Neural Networks (CNN) have recently seen tremendous success in various computer vision tasks. However, their application to problems with high dimensional input and output, such as high-resolution image and video segmentation or 3D medical imaging, has been limited by various factors. Primarily, in the training stage, it is necessary to store network activations for back propagation. In these settings, the memory requirements associated with storing activations can exceed what is feasible with current hardware, especially for problems in 3D. Motivated by the propagation of signal…
University of Nebraska（内布拉斯加大学）；Vanderbilt University（范德堡大学）；Stony Brook University（石溪大学）；Augusta University（奥古斯塔大学）
Convolutional Neural Networks (CNN) have redefined the state-of-the-art in many real-world applications, such as facial recognition, image classification, human pose estimation, and semantic segmentation. Despite their success, CNNs are vulnerable to adversarial attacks, where slight changes to their inputs may lead to sharp changes in their output in even well-trained networks. Set-based analysis methods can detect or prove the absence of bounded adversarial attacks, which can then be used to evaluate the effectiveness of neural network training methodology. Unfortunately, existing verificati…
Multi-view Hybrid Graph Convolutional Network for Volume-to-mesh Reconstruction in Cardiovascular MRI
Institute for Signals, Systems, and Computational Intelligence, sinc(i) CONICET-UNL（信号、系统与计算智能研究所，sinc(i) CONICET-UNL）；Centre for Computational Imaging and Simulation Technologies in Biomedicine (CISTIB), School of Computing, University of Leeds（生物医学计算成像与模拟技术中心（CISTIB），计算学院，莱斯特大学）；Christabel Pankhurst Institute, The University of Manchester（克里斯特贝尔·潘克赫斯特研究所，曼彻斯特大学）；Centre for Computational Imaging and Modelling in Medicine (CIMIM), Department of Computer Science, School of Engineering, and Division of Informatics Imaging and Data Science, School of Health Sciences, The University of Manchester（医学计算成像与建模中心（CIMIM），计算机科学系，工程学院，信息学成像与数据科学系，健康科学学院，曼彻斯特大学）；Medical Imaging Research Centre (MIRC), Department of Cardiovascular Sciences, KU Leuven（医学影像研究中心（MIRC），心血管科学系，鲁汶大学）
Activation in deep neural networks is fundamental to achieving non-linear mappings. Traditional studies mainly focus on finding fixed activations for a particular set of learning tasks or model architectures. The research on flexible activation is quite limited in both designing philosophy and application scenarios. In this study, three principles of choosing flexible activation components are proposed and a general combined form of flexible activation functions is implemented. Based on this, a novel family of flexible activation functions that can replace sigmoid or tanh in LSTM cells are imp…
From ACR O-RADS 2022 to Explainable Deep Learning: Comparative Performance of Expert Radiologists, Convolutional Neural Networks, Vision Transformers, and Fusion Models in Ovarian Masses
Danube Private University（多瑙河私立大学）；Urmia University of Medical Science（乌尔米耶医科大学）；Universiti Malaya（马来亚大学）；Universiti Teknologi MARA Selangor（玛拉工艺大学雪兰莪分校）；University of Southern Queensland（南昆士兰大学）；Austrian Center for Medical Innovation and Technology（奥地利医学创新与技术中心）；Medical University of Vienna（维也纳医科大学）
