首页/论文/1902.06550

arXiv 1902.065502019-02cs.CVcs.LG

LocalNorm: Robust Image Classification through Dynamically Regularized Normalization

Centrum Wiskunde & Informatica (CWI)（荷兰数学与计算机科学研究中心）；Holst Centre / IMEC（霍尔斯特中心/比利时微电子研究中心）；Technical University Eindhoven（埃因霍温理工大学）；University of Amsterdam（阿姆斯特丹大学）；Rijksuniversiteit Groningen（格罗宁根大学）

Bojian Yin, Siebren Schaafsma, Henk Corporaal, H. Steven Scholte, Sander M. Bohte

备注：: 14 pages, 17 figures

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

英文摘要相关论文

英文摘要：

While modern convolutional neural networks achieve outstanding accuracy on many image classification tasks, they are, compared to humans, much more sensitive to image degradation. Here, we describe a variant of Batch Normalization, LocalNorm, that regularizes the normalization layer in the spirit of Dropout while dynamically adapting to the local image intensity and contrast at test-time. We show that the resulting deep neural networks are much more resistant to noise-induced image degradation, improving accuracy by up to three times, while achieving the same or slightly better accuracy on non-degraded classical benchmarks. In computational terms, LocalNorm adds negligible training cost and little or no cost at inference time, and can be applied to already-trained networks in a straightforward manner.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

Effectiveness of Scaled Exponentially-Regularized Linear Units (SERLUs)

University of Technology Sydney（悉尼科技大学）；Qamcom Research and Technology AB（Qamcom研究与技术公司）

Recently, self-normalizing neural networks (SNNs) have been proposed with the intention to avoid batch or weight normalization. The key step in SNNs is to properly scale the exponential linear unit (referred to as SELU) to inherently incorporate normalization based on central limit theory. SELU is a monotonically increasing function, where it has an approximately constant negative output for large negative input. In this work, we propose a new activation function to break the monotonicity property of SELU while still preserving the self-normalizing property. Differently from SELU, the new func…

2018-07-27cs.AIcs.LGstat.ML

Why Regularized Auto-Encoders learn Sparse Representation?

SUNY Buffalo（纽约州立大学布法罗分校）

While the authors of Batch Normalization (BN) identify and address an important problem involved in training deep networks-- \textit{Internal Covariate Shift}-- the current solution has certain drawbacks. For instance, BN depends on batch statistics for layerwise input normalization during training which makes the estimates of mean and standard deviation of input (distribution) to hidden layers inaccurate due to shifting parameter values (especially during initial training epochs). Another fundamental problem with BN is that it cannot be used with batch-size $ 1 $ during training. We address t…

2016-06-17cs.CVcs.LGstat.ML

Manifold Regularized Deep Neural Networks using Adversarial Examples

Seoul National University（首尔国立大学）

Learning meaningful representations using deep neural networks involves designing efficient training schemes and well-structured networks. Currently, the method of stochastic gradient descent that has a momentum with dropout is one of the most popular training protocols. Based on that, more advanced methods (i.e., Maxout and Batch Normalization) have been proposed in recent years, but most still suffer from performance degradation caused by small perturbations, also known as adversarial examples. To address this issue, we propose manifold regularized networks (MRnet) that utilize a novel train…

2016-01-14cs.CVcs.LG

Feed-forward Uncertainty Propagation in Belief and Neural Networks

Czech Technical University in Prague（布拉格捷克理工大学）

We propose a feed-forward inference method applicable to belief and neural networks. In a belief network, the method estimates an approximate factorized posterior of all hidden units given the input. In neural networks the method propagates uncertainty of the input through all the layers. In neural networks with injected noise, the method analytically takes into account uncertainties resulting from this noise. Such feed-forward analytic propagation is differentiable in parameters and can be trained end-to-end. Compared to standard NN, which can be viewed as propagating only the means, we propa…

2018-11-01cs.CVcs.LGcs.NE

BN-invariant sharpness regularizes the training model to better generalization

University of Chinese Academy of Sciences（中国科学院大学）；Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院）；Microsoft Research（微软研究院）

It is arguably believed that flatter minima can generalize better. However, it has been pointed out that the usual definitions of sharpness, which consider either the maxima or the integral of loss over a $δ$ ball of parameters around minima, cannot give consistent measurement for scale invariant neural networks, e.g., networks with batch normalization layer. In this paper, we first propose a measure of sharpness, BN-Sharpness, which gives consistent value for equivalent networks under BN. It achieves the property of scale invariance by connecting the integral diameter with the scale of parame…

2021-01-08cs.LG

LapGM: A Multisequence MR Bias Correction and Normalization Model

University of California, Los Angeles（加州大学洛杉矶分校）；University of Victoria（维多利亚大学）；Brigham and Women’s Hospital（布莱根妇女医院）；Harvard Medical School（哈佛医学院）

A spatially regularized Gaussian mixture model, LapGM, is proposed for the bias field correction and magnetic resonance normalization problem. The proposed spatial regularizer gives practitioners fine-tuned control between balancing bias field removal and preserving image contrast preservation for multi-sequence, magnetic resonance images. The fitted Gaussian parameters of LapGM serve as control values which can be used to normalize image intensities across different patient scans. LapGM is compared to well-known debiasing algorithm N4ITK in both the single and multi-sequence setting. As a nor…

2022-09-27cs.CVphysics.med-phstat.AP

更多相关论文收起更多论文

Regularized Discrete Optimal Transport

CEREMADE, Université Paris-Dauphine（CEREMADE，巴黎第九大学）；IMB, UMR 5251, Université Bordeaux 1（IMB，波尔多第一大学）；Institut Universitaire de France（法国大学研究院）

This article introduces a generalization of the discrete optimal transport, with applications to color image manipulations. This new formulation includes a relaxation of the mass conservation constraint and a regularization term. These two features are crucial for image processing tasks, which necessitate to take into account families of multimodal histograms, with large mass variation across modes. The corresponding relaxed and regularized transportation problem is the solution of a convex optimization problem. Depending on the regularization used, this minimization can be solved using standa…

2013-07-21cs.CVcs.DMmath.OC

COP: Customized Deep Model Compression via Regularized Correlation-Based Filter-Level Pruning

Zhejiang University（浙江大学）；Fabu Inc.（法布公司）；Alibaba-Zhejiang University Joint Institute of Frontier Technologies（阿里巴巴-浙江大学前沿技术联合研究院）；GAC R&D Center（广汽研究院）

Neural network compression empowers the effective yet unwieldy deep convolutional neural networks (CNN) to be deployed in resource-constrained scenarios. Most state-of-the-art approaches prune the model in filter-level according to the "importance" of filters. Despite their success, we notice they suffer from at least two of the following problems: 1) The redundancy among filters is not considered because the importance is evaluated independently. 2) Cross-layer filter comparison is unachievable since the importance is defined locally within each layer. Consequently, we must manually specify l…

2019-06-25cs.CVcs.LG

Consistency Regularization for Generative Adversarial Networks

Google Research（谷歌研究院）

Generative Adversarial Networks (GANs) are known to be difficult to train, despite considerable research effort. Several regularization techniques for stabilizing training have been proposed, but they introduce non-trivial computational overheads and interact poorly with existing techniques like spectral normalization. In this work, we propose a simple, effective training stabilizer based on the notion of consistency regularization---a popular technique in the semi-supervised learning literature. In particular, we augment data passing into the GAN discriminator and penalize the sensitivity of …

2020-02-18cs.CVcs.LGstat.ML

DNGaussian：利用全局-局部深度归一化优化稀疏视图3D高斯辐射场

DNGaussian: Optimizing Sparse-View 3D Gaussian Radiance Fields with Global-Local Depth Normalization

Beihang University（北京航空航天大学）；Institute of Semiconductors, Chinese Academy of Sciences（中国科学院半导体研究所）；Griffith University（格里菲斯大学）；RIKEN AIP（理化学研究所人工智能项目）；The University of Tokyo（东京大学）

本文提出DNGaussian，一种基于3D高斯辐射场的深度正则化框架，通过硬软深度正则化与全局-局部深度归一化解决稀疏视图下的几何退化问题，实现低成本、高质量的实时少样本新视图合成。

2024-03-24cs.CV

SugarcaneNet：用于准确病害分类的LASSO正则化预训练模型优化集成

SugarcaneNet: An Optimized Ensemble of LASSO-Regularized Pre-trained Models for Accurate Disease Classification

Dhaka University of Engineering & Technology（达卡工程技术大学）；Jashore University of Science and Technology（杰索尔科技大学）；International Islamic University Chittagong（吉大港国际伊斯兰大学）；Zarqa University（扎尔卡大学）；Northumbria University（诺森比亚大学）；Qassim University（卡西姆大学）

研究提出sugarcaneNet2024模型，通过集成七个LASSO正则化预训练模型并结合网格搜索优化加权平均，实现甘蔗叶片病害的高精度分类检测。

2024-11-16cs.AIcs.CVcs.LG

An ELU Network with Total Variation for Image Denoising

In this paper, we propose a novel convolutional neural network (CNN) for image denoising, which uses exponential linear unit (ELU) as the activation function. We investigate the suitability by analyzing ELU's connection with trainable nonlinear reaction diffusion model (TNRD) and residual denoising. On the other hand, batch normalization (BN) is indispensable for residual denoising and convergence purpose. However, direct stacking of BN and ELU degrades the performance of CNN. To mitigate this issue, we design an innovative combination of activation layer and normalization layer to exploit and…

2017-08-14cs.CV

SRAGAN：用于中国水墨画风格迁移的显著性正则化与注意力生成对抗网络

SRAGAN: Saliency Regularized and Attended Generative Adversarial Network for Chinese Ink-wash Painting Style Transfer

Wangxuan Institute of Computer Technology, Peking University（北京大学王轩计算机技术研究所）；School of Mathematical Sciences, University of Chinese Academy of Sciences（中国科学院大学数学科学学院）

本文提出SRAGAN解决中国水墨画风格迁移中内容细节易被破坏的问题，通过引入显著性检测提出SIOU损失和SANorm正则化内容结构，并设计显著性注意力判别器增强笔触与纹理，实验证明其优于GAN和扩散模型等先进方法。

2025-10-31cs.CV

TurboT2VA：基于分数正则化一致性蒸馏的快速大规模文本-视频-音频生成

TurboT2VA: Fast Large-Scale Text-to-Video-Audio Generation via Score-Regularized Consistency Distillation

Zhejiang University（浙江大学）；Tianjin University（天津大学）；Tsinghua University（清华大学）；Qingdao University（青岛大学）；National University of Singapore（新加坡国立大学）

本文提出TurboT2VA框架，通过多模态归一化与渐进式课程方案，在保持音视频生成质量的同时，大幅加速190亿参数联合T2VA模型的推理，在不同分辨率下实现最高54.67倍的生成器加速。

2026-09-10cs.CV

Orthogonal Deep Neural Networks

South China University of Technology（华南理工大学）；The University of Sydney（悉尼大学）

In this paper, we introduce the algorithms of Orthogonal Deep Neural Networks (OrthDNNs) to connect with recent interest of spectrally regularized deep learning methods. OrthDNNs are theoretically motivated by generalization analysis of modern DNNs, with the aim to find solution properties of network weights that guarantee better generalization. To this end, we first prove that DNNs are of local isometry on data distributions of practical interest; by using a new covering of the sample space and introducing the local isometry property of DNNs into generalization analysis, we establish a new ge…

2019-10-15cs.CVcs.LGstat.ML

反向卷积及其在图像恢复中的应用

Reverse Convolution and Its Applications to Image Restoration

Nanjing University（南京大学）；The Hong Kong Polytechnic University（香港理工大学）；OPPO Research Institute（OPPO研究院）

提出深度反向卷积算子通过正则化最小二乘优化实现卷积逆运算，构建类Transformer反向卷积块组成ConverseNet，在图像去噪、超分辨率和去模糊任务中验证了其有效性。

2025-08-15cs.CV