There exists a distribution discrepancy between training and testing, in the way images are fed to modern CNNs. Recent work tried to bridge this gap either by fine-tuning or re-training the network at different resolutions. However re-training a network is rarely cheap and not always viable. To this end, we propose a simple solution to address the train-test distributional shift and enhance the performance of pre-trained models -- which commonly ship as a package with deep learning platforms \eg, PyTorch. Specifically, we demonstrate that running inference on the center crop of an image is not always the best as important discriminatory information may be cropped-off. Instead we propose to combine results for multiple random crops for a test image. This not only matches the train time augmentation but also provides the full coverage of the input image. We explore combining representation of random crops through averaging at different levels \ie, deep feature level, logit level, and softmax level. We demonstrate that, for various families of modern deep networks, such averaging results in better validation accuracy compared to using a single central crop per image. The softmax averaging results in the best performance for various pre-trained networks without requiring any re-training or fine-tuning whatsoever. On modern GPUs with batch processing, the paper's approach to inference of pre-trained networks, is essentially free as all images in a batch can all be processed at once.
National University of Sciences and Technology (NUST)（国立科技大学）
Table Structure Recognition is an essential part of end-to-end tabular data extraction in document images. The recent success of deep learning model architectures in computer vision remains to be non-reflective in table structure recognition, largely because extensive datasets for this domain are still unavailable while labeling new data is expensive and time-consuming. Traditionally, in computer vision, these challenges are addressed by standard augmentation techniques that are based on image transformations like color jittering and random cropping. As demonstrated by our experiments, these t…
Convolutional neural networks (CNN) are capable of learning robust representation with different regularization methods and activations as convolutional layers are spatially correlated. Based on this property, a large variety of regional dropout strategies have been proposed, such as Cutout, DropBlock, CutMix, etc. These methods aim to promote the network to generalize better by partially occluding the discriminative parts of objects. However, all of them perform this operation randomly, without capturing the most important region(s) within an object. In this paper, we propose Attentive CutMix…
EventZoom: A Progressive Approach to Event-Based Data Augmentation for Enhanced Neuromorphic Vision
School of Future Technology, University of Chinese Academy of Sciences（中国科学院大学未来技术学院）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）
In this paper we propose the use of image pixel position coordinate system to improve image classification accuracy in various applications. Specifically, we hypothesize that the use of pixel coordinates will lead to (a) Resolution invariant performance. Here, by resolution we mean the spacing between the pixels rather than the size of the image matrix. (b) Overall improvement in classification accuracy in comparison with network models trained without local pixel coordinates. This is due to position coordinates enabling the network to learn relationship between parts of objects, mimicking the…
University of Waterloo（滑铁卢大学）；McGill University（麦吉尔大学）；St. Mary’s Hospital（圣玛丽医院）；Waterloo AI Institute（滑铁卢人工智能研究院）；DarwinAI Corp.（达尔文人工智能公司）
As the global population continues to face significant negative impact by the on-going COVID-19 pandemic, there has been an increasing usage of point-of-care ultrasound (POCUS) imaging as a low-cost and effective imaging modality of choice in the COVID-19 clinical workflow. A major barrier with widespread adoption of POCUS in the COVID-19 clinical workflow is the scarcity of expert clinicians that can interpret POCUS examinations, leading to considerable interest in deep learning-driven clinical decision support systems to tackle this challenge. A major challenge to building deep neural networ…
Diffusion Model-Based Data Augmentation for Enhanced Neuron Segmentation
State Key Laboratory of Brain Cognition（脑认知国家重点实验室）；Brain-inspired Intelligence Technology, Institute of Automation, Chinese Academy of Sciences（脑启发智能技术，自动化研究所，中国科学院）；School of Future Technology, University of Chinese Academy of Sciences（未来技术学院，中国科学院大学）；School of Artificial Intelligence, University of Chinese Academy of Sciences（人工智能学院，中国科学院大学）
University of New Orleans（新奥尔良大学）；US Army Corps of Engineers（美国陆军工程兵团）
Imbalanced datasets are a significant challenge in real-world scenarios. They lead to models that underperform on underrepresented classes, which is a critical issue in infrastructure inspection. This paper introduces the Enhanced Feature Pyramid Network (E-FPN), a deep learning model for the semantic segmentation of culverts and sewer pipes within imbalanced datasets. The E-FPN incorporates architectural innovations like sparsely connected blocks and depth-wise separable convolutions to improve feature extraction and handle object variations. To address dataset imbalance, the model employs st…
Diffusion-Enhanced Test-time Adaptation with Text and Image Augmentation
Institute of High Performance Computing (IHPC), Agency for Science, Technology and Research (A*STAR)（高性能计算研究所（新加坡科技研究局））；National University of Singapore（新加坡国立大学）；Harbin Institute of Technology（哈尔滨工业大学）；Mohamed bin Zayed University of Artificial Intelligence (MBZUAI)（穆罕默德·本·扎耶德人工智能大学）；Australian National University（澳大利亚国立大学）；Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））
Centralized Copy-Paste: Enhanced Data Augmentation Strategy for Wildland Fire Semantic Segmentation
Mechanical and Aerospace Engineering, AIAA Student Member（机械与航空航天工程，AIAA学生会员）；Department of Computer Science（计算机科学系）；School of Environment and Natural Resources（环境与自然资源学院）；Computer Science and Engineering（计算机科学与工程）；Department of Electronics and Telecommunications Engineering（电子与电信工程系）；Aeronautical and Astronautical Engineering, AIAA Associate Fellow（航空与航天工程，AIAA associate fellow）
In the realm of medical imaging, the training of machine learning models necessitates a large and varied training dataset to ensure robustness and interoperability. However, acquiring such diverse and heterogeneous data can be difficult due to the need for expert labeling of each image and privacy concerns associated with medical data. To circumvent these challenges, data augmentation has emerged as a promising and cost-effective technique for increasing the size and diversity of the training dataset. In this study, we provide a comprehensive review of the specific data augmentation techniques…
Memory-enhanced Retrieval Augmentation for Long Video Understanding
Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学光华学院人工智能学院）；Beijing Academy of Artificial Intelligence（北京人工智能研究院）；University of Trento（特伦托大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）
How we won BraTS 2023 Adult Glioma challenge? Just faking it! Enhanced Synthetic Data Augmentation and Model Ensemble for brain tumour segmentation
University of Minho（米尼奥大学）；Institute for Artificial Intelligence in Medicine（人工智能医学研究所）；Graz University of Technology（格拉茨工业大学）；University Medicine Essen（埃森大学医学中心）；Cancer Research Center Cologne Essen (CCCE)（科隆-埃森癌症研究中心）；University Hospital Essen（埃森大学医院）；German Cancer Consortium (DKTK)（德国癌症联盟）
Evaluation of Randomization through Style Transfer for Enhanced Domain Generalization
German Cancer Research Center (DKFZ)（德国癌症研究中心（DKFZ））；German Cancer Consortium (DKTK), partner site Frankfurt/Mainz, a partnership between DKFZ and UCT Frankfurt-Marburg（德国癌症联盟（DKTK），法兰克福/美因茨合作站点，由DKFZ与UCT法兰克福-马尔堡合作）；Goethe University Frankfurt, Department of Computer Science（法兰克福大学计算机科学系）；The Hessian Center for Artificial Intelligence (hessian.AI)（黑森人工智能中心（hessian.AI））
Synthetic Data Augmentation for Enhanced Chicken Carcass Instance Segmentation
Department of Biological and Agricultural Engineering, University of Arkansas（生物与农业工程系，阿肯色大学）；Department of Food Science, University of Arkansas（食品科学系，阿肯色大学）；Department of Electrical Engineering, University of South Florida（电气工程系，佛罗里达州立大学）；Department of Mechanical Engineering, University of Arkansas（机械工程系，阿肯色大学）；Department of Industrial Engineering, Purdue University（工业工程系，普渡大学）
New Mexico Institute of Mining and Technology（新墨西哥矿业技术学院）；Los Alamos National Laboratory（洛斯阿拉莫斯国家实验室）；Descartes Labs, Inc.（笛卡尔实验室公司）
Anomalous change detection (ACD) methods separate common, uninteresting changes from rare, significant changes in co-registered images collected at different points in time. In this paper we evaluate methods to improve the performance of ACD in detecting human activity in SAR imagery using outdoor music festivals as a target. Our results show that the low dimensionality of SAR data leads to poor performance of ACD when compared to simpler methods such as image differencing, but augmenting the dimensionality of our input feature space by incorporating local spatial information leads to enhanced…
A Multi-Stage Deep Learning Framework with PKCP-MixUp Augmentation for Pediatric Liver Tumor Diagnosis Using Multi-Phase Contrast-Enhanced CT
Wuhan Children's Hospital (Wuhan Maternal and Child Healthcare Hospital), Tongji Medical College, Huazhong University of Science and Technology（华中科技大学同济医学院武汉儿童医院（武汉市妇幼保健院））；Wuhan University（武汉大学）；Wuhan Cloud Computing Technology Co., Ltd.（武汉云计算技术有限公司）
Do Generative Metrics Predict YOLO Performance? An Evaluation Across Models, Augmentation Ratios, and Dataset Complexity
School of Mechanical and Mining Engineering, The University of Queensland,Level 4, Mansergh Shaw Building (45), St Lucia, QLD 4072, Australia（昆士兰大学机械与采矿工程学院）；Swinburne University of Technology, John Street, Hawthorn, VIC 3122, Australia（斯威本科技大学）
We proposed a new modeling method to promote the performance of prohibited items recognition via X-ray image. We analyzed the characteristics of prohibited items and X-ray images. We found the fact that the scales of some items are too small to be recognized which encumber the model performance. Then we adopted a set of data augmentation and modified the model to adapt the field of prohibited items recognition. The Convolutional Block Attention Module(CBAM) and rescoring mechanism has been assembled into the model. By the modification, our model achieved a mAP of 89.9% on SIXray10, mAP of 74.8…
Princeton University（普林斯顿大学）；Carnegie Mellon University（卡内基梅隆大学）；Institute for Advanced Study（高等研究院）
Recent research shows that for training with $\ell_2$ loss, convolutional neural networks (CNNs) whose width (number of channels in convolutional layers) goes to infinity correspond to regression with respect to the CNN Gaussian Process kernel (CNN-GP) if only the last layer is trained, and correspond to regression with respect to the Convolutional Neural Tangent Kernel (CNTK) if all layers are trained. An exact algorithm to compute CNTK (Arora et al., 2019) yielded the finding that classification accuracy of CNTK on CIFAR-10 is within 6-7% of that of that of the corresponding CNN architecture…
Mixed Sample Regularization (MSR), such as MixUp or CutMix, is a powerful data augmentation strategy to generalize convolutional neural networks. Previous empirical analysis has illustrated an orthogonal performance gain between MSR and conventional offline Knowledge Distillation (KD). To be more specific, student networks can be enhanced with the involvement of MSR in the training stage of sequential distillation. Yet, the interplay between MSR and online knowledge distillation, where an ensemble of peer students learn mutually from each other, remains unexplored. To bridge the gap, we make t…
Deep Neural Networks (DNNs) show a significant impact on medical imaging. One significant problem with adopting DNNs for skin cancer classification is that the class frequencies in the existing datasets are imbalanced. This problem hinders the training of robust and well-generalizing models. Data Augmentation addresses this by using existing data more effectively. However, standard data augmentation implementations are manually designed and produce only limited reasonably alternative data. Instead, Generative Adversarial Networks (GANs) is utilized to generate a much broader set of augmentatio…
Nowadays, commonly-used authentication systems for mobile device users, e.g. password checking, face recognition or fingerprint scanning, are susceptible to various kinds of attacks. In order to prevent some of the possible attacks, these explicit authentication systems can be enhanced by considering a two-factor authentication scheme, in which the second factor is an implicit authentication system based on analyzing motion sensor data captured by accelerometers or gyroscopes. In order to avoid any additional burdens to the user, the registration process of the implicit authentication system m…
Kakao Brain；MILA（米拉研究所）；Université de Montréal（蒙特利尔大学）
Data augmentation is an essential technique for improving generalization ability of deep learning models. Recently, AutoAugment has been proposed as an algorithm to automatically search for augmentation policies from a dataset and has significantly enhanced performances on many image recognition tasks. However, its search method requires thousands of GPU hours even for a relatively small dataset. In this paper, we propose an algorithm called Fast AutoAugment that finds effective augmentation policies via a more efficient search strategy based on density matching. In comparison to AutoAugment, …
