High-quality labeled data is essential to successfully train supervised machine learning models. Although a large amount of unlabeled data is present in the medical domain, labeling poses a major challenge: medical professionals who can expertly label the data are a scarce and expensive resource. Making matters worse, voxel-wise delineation of data (e.g. for segmentation tasks) is tedious and suffers from high inter-rater variance, thus dramatically limiting available training data. We propose a recursive training strategy to perform the task of semantic segmentation given only very few training samples with pixel-level annotations. We expand on this small training set having cheaper image-level annotations using a recursive training strategy. We apply this technique on the segmentation of intracranial hemorrhage (ICH) in CT (computed tomography) scans of the brain, where typically few annotated data is available.
We introduce an approach to training a given compact network. To this end, we leverage over-parameterization, which typically improves both neural network optimization and generalization. Specifically, we propose to expand each linear layer of the compact network into multiple consecutive linear layers, without adding any nonlinearity. As such, the resulting expanded network, or ExpandNet, can be contracted back to the compact one algebraically at inference. In particular, we introduce two convolutional expansion strategies and demonstrate their benefits on several tasks, including image class…
Exponentially Weighted Instance-Aware Repeat Factor Sampling for Long-Tailed Object Detection Model Training in Unmanned Aerial Vehicles Surveillance Scenarios
Center for Ubiquitous Computing (UBICOMP), University of Oulu, Finland（通用计算中心（UBICOMP），奥卢大学，芬兰）；Center for Machine Vision and Signal Analysis (CMVS), University of Oulu, Finland（机器视觉与信号分析中心（CMVS），奥卢大学，芬兰）；Center for Wireless Communications (CWC), University of Oulu, Finland（无线通信中心（CWC），奥卢大学，芬兰）；Ming Hsieh Department of Electrical and Computer Engineering, University of Southern California, United States（明斯希德电气与计算机工程系，南加州大学，美国）；Department of Computer Science, University of Helsinki, Finland（计算机科学系，赫尔辛基大学，芬兰）；University of Oulu, Oulu, Finland（奥卢大学，奥卢，芬兰）；University of Helsinki, Helsinki, Finland（赫尔辛基大学，赫尔辛基，芬兰）；University of Southern California, Los Angeles, United States（南加州大学，洛杉矶，美国）
We introduce a new loss function for the weakly-supervised training of semantic image segmentation models based on three guiding principles: to seed with weak localization cues, to expand objects based on the information about which classes can occur in an image, and to constrain the segmentations to coincide with object boundaries. We show experimentally that training a deep convolutional neural network using the proposed loss function leads to substantially better segmentations than previous state-of-the-art methods on the challenging PASCAL VOC 2012 dataset. We furthermore give insight into…
DART: Diversify-Aggregate-Repeat Training Improves Generalization of Neural Networks
Indian Institute of Technology, Varanasi（瓦拉纳西印度理工学院）；Indian Institute of Technology, Dhanbad（丹巴德印度理工学院）；Indian Institute of Science, Bangalore（班加罗尔印度科学学院）
University of Pavia（帕维亚大学）；Deutsches Herzzentrum Berlin（柏林德国心脏中心）；Charité – Universitätsmedizin Berlin（柏林夏里特医学院）；Berlin Institute of Health (BIH)（柏林健康研究所）；DZHK (German Centre for Cardiovascular Research)（德国心血管研究中心）；University of Verona（维罗纳大学）；University of Parma（帕尔马大学）；Sant’Anna Hospital（圣安娜医院）；Regina Montis Regalis Hospital（蒙蒂斯雷加利斯女王医院）
Heart failure (HF) affects at least 26 million people worldwide, so predicting adverse events in HF patients represents a major target of clinical data science. However, achieving large sample sizes sometimes represents a challenge due to difficulties in patient recruiting and long follow-up times, increasing the problem of missing data. To overcome the issue of a narrow dataset cardinality (in a clinical dataset, the cardinality is the number of patients in that dataset), population-enhancing algorithms are therefore crucial. The aim of this study was to design a random shuffle method to enha…
Tel Aviv University（特拉维夫大学）；axilon；axilion；The Interdisciplinary Center（跨学科研究中心）
We present a simple and intuitive Focus-and-eXpand (\fax) method to guide the training process of a neural network towards a specific solution. Optimizing a neural network is a highly non-convex problem. Typically, the space of solutions is large, with numerous possible local minima, where reaching a specific minimum depends on many factors. In many cases, however, a solution which considers specific aspects, or features, of the input is desired. For example, in the presence of bias, a solution that disregards the biased feature is a more robust and accurate one. Drawing inspiration from Param…
In many deep reinforcement learning settings, when an agent takes an action, it repeats the same action a predefined number of times without observing the states until the next action-decision point. This technique of action repetition has several merits in training the agent, but the data between action-decision points (i.e., intermediate frames) are, in effect, discarded. Since the amount of training data is inversely proportional to the interval of action repeats, they can have a negative impact on the sample efficiency of training. In this paper, we propose a simple but effective approach …
Despite remarkable success in practice, modern machine learning models have been found to be susceptible to adversarial attacks that make human-imperceptible perturbations to the data, but result in serious and potentially dangerous prediction errors. To address this issue, practitioners often use adversarial training to learn models that are robust against such attacks at the cost of higher generalization error on unperturbed test sets. The conventional wisdom is that more training data should shrink the gap between the generalization error of adversarially-trained models and standard models.…
Anomaly detection (AD), separating anomalies from normal data, has many applications across domains, from security to healthcare. While most previous works were shown to be effective for cases with fully or partially labeled data, that setting is in practice less common due to labeling being particularly tedious for this task. In this paper, we focus on fully unsupervised AD, in which the entire training dataset, containing both normal and anomalous samples, is unlabeled. To tackle this problem effectively, we propose to improve the robustness of one-class classification trained on self-superv…
Naval Research Laboratory（海军研究实验室）；University of Maryland（马里兰大学）
We introduce the concept of dynamically growing a neural network during training. In particular, an untrainable deep network starts as a trainable shallow network and newly added layers are slowly, organically added during training, thereby increasing the network's depth. This is accomplished by a new layer, which we call DropIn. The DropIn layer starts by passing the output from a previous layer (effectively skipping over the newly added layers), then increasingly including units from the new layers for both feedforward and backpropagation. We show that deep networks, which are untrainable wi…
Indian Institute of Technology（印度理工学院）；Ramakrishna Mission Ashrama（罗摩克里希纳传教修道院）；Rajalakshmi Children Foundation（拉贾拉克什米儿童基金会）
GANs have two competing modules: the generator module is trained to generate new examples, and the discriminator module is trained to discriminate real examples from generated examples. The training procedure of GAN is modeled as a finitely repeated simultaneous game. Each module tries to increase its performance at every repetition of the base game (at every batch of training data) in a non-cooperative manner. We observed that each module can perform better and learn faster if training is modeled as an infinitely repeated simultaneous game. At every repetition of the base game (at every batch…
