We propose a new notion of `non-linearity' of a network layer with respect to an input batch that is based on its proximity to a linear system, which is reflected in the non-negative rank of the activation matrix. We measure this non-linearity by applying non-negative factorization to the activation matrix. Considering batches of similar samples, we find that high non-linearity in deep layers is indicative of memorization. Furthermore, by applying our approach layer-by-layer, we find that the mechanism for memorization consists of distinct phases. We perform experiments on fully-connected and convolutional neural networks trained on several image and audio datasets. Our results demonstrate that as an indicator for memorization, our technique can be used to perform early stopping.
We study finite sample expressivity, i.e., memorization power of ReLU networks. Recent results require $N$ hidden nodes to memorize/interpolate arbitrary $N$ data points. In contrast, by exploiting depth, we show that 3-layer ReLU networks with $Ω(\sqrt{N})$ hidden nodes can perfectly memorize most datasets with $N$ points. We also prove that width $Θ(\sqrt{N})$ is necessary and sufficient for memorizing $N$ data points, proving tight bounds on memorization capacity. The sufficiency result can be extended to deeper networks; we show that an $L$-layer network with $W$ parameters in the hidden l…
We study the memorization power of feedforward ReLU neural networks. We show that such networks can memorize any $N$ points that satisfy a mild separability assumption using $\tilde{O}\left(\sqrt{N}\right)$ parameters. Known VC-dimension upper bounds imply that memorizing $N$ samples requires $Ω(\sqrt{N})$ parameters, and hence our construction is optimal up to logarithmic factors. We also give a generalized construction for networks with depth bounded by $1 \leq L \leq \sqrt{N}$, for memorizing $N$ samples using $\tilde{O}(N/L)$ parameters. This bound is also optimal up to logarithmic factors…
Microsoft Research（微软研究院）；Weizmann Institute（魏茨曼科学研究所）；University of Washington（华盛顿大学）
In 1988, Eric B. Baum showed that two-layers neural networks with threshold activation function can perfectly memorize the binary labels of $n$ points in general position in $\mathbb{R}^d$ using only $\ulcorner n/d \urcorner$ neurons. We observe that with ReLU networks, using four times as many neurons one can fit arbitrary real labels. Moreover, for approximate memorization up to error $ε$, the neural tangent kernel can also memorize with only $O\left(\frac{n}{d} \cdot \log(1/ε) \right)$ neurons (assuming that the data is well dispersed too). We show however that these constructions give rise…
Memorization capacity of deep ReLU neural networks characterized by width and depth
School of Mathematics (Zhuhai), Sun Yat-sen University, Zhuhai, China（中山大学数学学院（珠海）、中山大学珠海校区，中国）；School of Mathematics (Zhuhai) and Guangdong Province Key Laboratory of Computational Science, Sun Yat-sen University, Zhuhai, China（中山大学数学学院（珠海）和广东省计算科学重点实验室，中山大学珠海校区，中国）
The roles played by learning and memorization represent an important topic in deep learning research. Recent work on this subject has shown that the optimization behavior of DNNs trained on shuffled labels is qualitatively different from DNNs trained with real labels. Here, we propose a novel permutation approach that can differentiate memorization from learning in deep neural networks (DNNs) trained as usual (i.e., using the real labels to guide the learning, rather than shuffled labels). The evaluation of weather the DNN has learned and/or memorized, happens in a separate step where we compa…
Many empirical studies have demonstrated the performance benefits of conditional computation in neural networks, including reduced inference time and power consumption. We study the fundamental limits of neural conditional computation from the perspective of memorization capacity. For Rectified Linear Unit (ReLU) networks without conditional computation, it is known that memorizing a collection of $n$ input-output relationships can be accomplished via a neural network with $O(\sqrt{n})$ neurons. Calculating the output of this neural network can be accomplished using $O(\sqrt{n})$ elementary ar…
Google Research（谷歌研究院）；Boston University（波士顿大学）；Blueshift, Alphabet（Alphabet 旗下 Blueshift）
We construct an experimental setup in which changing the scale of initialization strongly impacts the implicit regularization induced by SGD, interpolating from good generalization performance to completely memorizing the training set while making little progress on the test set. Moreover, we find that the extent and manner in which generalization ability is affected depends on the activation and loss function used, with $\sin$ activation demonstrating extreme memorization. In the case of the homogeneous ReLU activation, we show that this behavior can be attributed to the loss function. Our em…
End-to-end (E2E) models are often being accompanied by language models (LMs) via shallow fusion for boosting their overall quality as well as recognition of rare words. At the same time, several prior works show that LMs are susceptible to unintentionally memorizing rare or unique sequences in the training data. In this work, we design a framework for detecting memorization of random textual sequences (which we call canaries) in the LM training data when one has only black-box (query) access to LM-fused speech recognizer, as opposed to direct access to the LM. On a production-grade Conformer R…
We develop a corrective mechanism for neural network approximation: the total available non-linear units are divided into multiple groups and the first group approximates the function under consideration, the second group approximates the error in approximation produced by the first group and corrects it, the third group approximates the error produced by the first and second groups together and so on. This technique yields several new representation and learning results for neural networks. First, we show that two-layer neural networks in the random features regime (RF) can memorize arbitrary…
ReLU neural-networks have been in the focus of many recent theoretical works, trying to explain their empirical success. Nonetheless, there is still a gap between current theoretical results and empirical observations, even in the case of shallow (one hidden-layer) networks. For example, in the task of memorizing a random sample of size $m$ and dimension $d$, the best theoretical result requires the size of the network to be $\tildeΩ(\frac{m^2}{d})$, while empirically a network of size slightly larger than $\frac{m}{d}$ is sufficient. To bridge this gap, we turn to study a simplified model for…
Facebook AI Research（Facebook人工智能研究院）；Texas A&M University（得克萨斯农工大学）；University of Pennsylvania（宾夕法尼亚大学）
The success of deep networks has been attributed in part to their expressivity: per parameter, deep networks can approximate a richer class of functions than shallow networks. In ReLU networks, the number of activation patterns is one measure of expressivity; and the maximum number of patterns grows exponentially with the depth. However, recent work has showed that the practical expressivity of deep networks - the functions they can learn rather than express - is often far from the theoretical maximum. In this paper, we show that the average number of activation patterns for ReLU networks at i…
MPI-MIS（马克斯·普朗克数学研究所）；Institute of Science and Technology Austria (ISTA)（奥地利科学技术研究所）；UCLA（加州大学洛杉矶分校）
A recent line of work has analyzed the theoretical properties of deep neural networks via the Neural Tangent Kernel (NTK). In particular, the smallest eigenvalue of the NTK has been related to the memorization capacity, the global convergence of gradient descent algorithms and the generalization of deep nets. However, existing results either provide bounds in the two-layer setting or assume that the spectrum of the NTK matrices is bounded away from 0 for multi-layer networks. In this paper, we provide tight bounds on the smallest eigenvalue of NTK matrices for deep ReLU nets, both in the limit…
Normandie Univ（诺曼底大学）；UNICAEN（卡昂大学）；ENSICAEN（卡昂高等工程师学院）；CNRS — UMR GREYC（法国国家科学研究中心 — 灰谐实验室联合研究单位（注：CNRS为法国国家科学研究中心，UMR为联合研究单位缩写，GREYC为实验室名，此处按原文提取机构部分））
State of the art deep generative networks are capable of producing images with such incredible realism that they can be suspected of memorizing training images. It is why it is not uncommon to include visualizations of training set nearest neighbors, to suggest generated images are not simply memorized. We demonstrate this is not sufficient and motivates the need to study memorization/overfitting of deep generators with more scrutiny. This paper addresses this question by i) showing how simple losses are highly effective at reconstructing images for deep generators ii) analyzing the statistics…
Exploring prompts to elicit memorization in masked language model-based named entity recognition
University of Vienna（维也纳大学）；Faculty of Computer Science, University of Vienna（维也纳大学计算机科学学院）；UniVie Doctoral School Computer Science（维也纳大学计算机科学博士学院）；Faculty of Philological and Cultural Studies, University of Vienna（维也纳大学语言与文化研究学院）
Central Institute of Mental Health, Mannheim, Germany（德国曼海姆中央心理健康研究所）；Heidelberg University, Heidelberg, Germany（德国海德堡大学）；Interdisciplinary Center for Scientific Computing, Heidelberg University, Germany（德国海德堡大学跨学科科学计算中心）；Faculty of Mathematics and Computer Science, Heidelberg University, Heidelberg, Germany（德国海德堡大学数学与计算机科学学院）
Measures of Information Reflect Memorization Patterns
Delhi Technological University（德里理工大学）；Google Research India（谷歌研究院印度分部）；Amazon Web Services（亚马逊网络服务）；Technion – Israel Institute of Technology（以色列理工学院）；Carnegie Mellon University（卡内基梅隆大学）
Applicability of memorization indicators for early spotting of overfitting while recalibrating sEMG-decoders on low sample sizes
German Aerospace Center (DLR), Institute for AI Safety and Security（德国航空航天中心（DLR），人工智能安全与安全研究所）；Faculty of Electrical Engineering and Information Technology, Ruhr University Bochum（波鸿鲁尔大学电气工程与信息技术学院）；Institute for Neural Computation, Ruhr University Bochum（波鸿鲁尔大学神经计算研究所）；Institute of Computer Science, Ruhr West University of Applied Science（鲁尔西部应用科学大学计算机科学研究所）
