首页/论文/2011.13052

arXiv 2011.130522020-11-25cs.LG

Squared $\ell_2$ Norm as Consistency Loss for Leveraging Augmented Data to Learn Robust and Invariant Representations

Carnegie Mellon University（卡内基梅隆大学）

Haohan Wang, Zeyi Huang, Xindi Wu, Eric P. Xing

备注：: 12 pages and an additional 9 pages as appendix

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

英文摘要相关论文

英文摘要：

Data augmentation is one of the most popular techniques for improving the robustness of neural networks. In addition to directly training the model with original samples and augmented samples, a torrent of methods regularizing the distance between embeddings/representations of the original samples and their augmented counterparts have been introduced. In this paper, we explore these various regularization choices, seeking to provide a general understanding of how we should regularize the embeddings. Our analysis suggests the ideal choices of regularization correspond to various assumptions. With an invariance test, we argue that regularization is important if the model is to be used in a broader context than the accuracy-driven setting because non-regularized approaches are limited in learning the concept of invariance, despite equally high accuracy. Finally, we also show that the generic approach we identified (squared $\ell_2$ norm regularized augmentation) outperforms several recent methods, which are each specially designed for one task and significantly more complicated than ours, over three different tasks.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

外部引导何时有助于大语言模型推理？引导增强GRPO的偏差-方差理论

When Does External Guidance Help LLM Reasoning? A Bias-Variance Theory of Guidance-Augmented GRPO

Federal University of Bahia（巴伊亚联邦大学）

提出GA-GRPO统一框架，将外部引导建模为随机算子，给出收敛速率与最优权重闭式解，实验证明其性能优于现有方法且节省31%计算资源。

2026-10-07cs.CLcs.LG

用于（非平方）欧氏距离的 $k$-PCA：多项式时间近似

$k$-PCA for (non-squared) Euclidean Distances: Polynomial Time Approximation

Robotics & Big Data Labs, University of Haifa, Israel（机器人与大数据实验室，海法大学）

本文针对欧氏空间中 $k$-子空间中位数（非平方距离和最小化）问题，提出首个运行时间与近似因子均不随 $k$ 指数增长的多项式时间确定性算法，近似因子为 $\sqrt{d}$，并给出代码与实验验证。

2025-07-19cs.CGcs.DScs.LG

损失增强知识追踪综述

Survey of Loss Augmented Knowledge Tracing

University of Virginia（弗吉尼亚大学）

本文综述了使用高级损失函数（如正则化和对比学习）训练的深度学习知识追踪算法，对比了Bi-CLKT等方法的性能，并展望了混合损失与上下文感知建模的未来方向。

2025-04-21cs.LG

Efficient Elastic Net Regularization for Sparse Linear Models

University of California, San Diego（加州大学圣地亚哥分校）

This paper presents an algorithm for efficient training of sparse linear models with elastic net regularization. Extending previous work on delayed updates, the new algorithm applies stochastic gradient updates to non-zero features only, bringing weights current as needed with closed-form updates. Closed-form delayed updates for the $\ell_1$, $\ell_{\infty}$, and rarely used $\ell_2$ regularizers have been described previously. This paper provides closed-form updates for the popular squared norm $\ell^2_2$ and elastic net regularizers. We provide dynamic programming algorithms that perform eac…

2015-07-02cs.LG

使用降维的学习增强K-Means聚类

Learning-Augmented K-Means Clustering Using Dimensional Reduction

University of Brawijaya（布拉维贾亚大学）

针对k-means算法计算成本随k值增加且易陷入局部最小值的挑战，提出利用主成分分析（PCA）进行降维的学习增强方案，在k值为10和25时有效降低了成本。

2024-01-06cs.LG

用于半正定矩阵的快速长度平方采样

Fast Length-Squared Sampling for Positive-Semidefinite Matrices

University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）；University of California Berkeley（加州大学伯克利分校）

本研究提出一种期望O(n)时间的半正定矩阵快速长度平方采样算法，消除了需获取矩阵列范数的假设，可应用于鲁棒半正定低秩近似等问题，效果接近文献中的复杂方法。

2026-08-14cs.DScs.LG

更多相关论文收起更多论文

$\ell_1$-regression with Heavy-tailed Distributions

Nanjing University（南京大学）

In this paper, we consider the problem of linear regression with heavy-tailed distributions. Different from previous studies that use the squared loss to measure the performance, we choose the absolute loss, which is capable of estimating the conditional median. To address the challenge that both the input and output could be heavy-tailed, we propose a truncated minimization problem, and demonstrate that it enjoys an $\widetilde{O}(\sqrt{d/n})$ excess risk, where $d$ is the dimensionality and $n$ is the number of samples. Compared with traditional work on $\ell_1$-regression, the main advantag…

2018-10-25cs.LGstat.ML

结合自编码器和增强数据的混合深度卷积神经网络用于预测查找表 2006

Hybrid Deep Convolutional Neural Networks Combined with Autoencoders And Augmented Data To Predict The Look-Up Table 2006

Mohamed Khider University（穆罕默德·希德尔大学）；Norwegian University of Science and Technology（挪威科技大学）；Oak Ridge National Laboratory（橡树岭国家实验室）；Gulf University for Science & Technology（海湾科技大学）

本研究提出一种结合自编码器和数据增强的混合深度卷积神经网络模型，通过特征增强显著提升了临界热通量（CHF）的预测精度，在7225个样本上测试R2达0.9826，优于基础模型。

2024-08-26cs.AIcs.LG

On Dropout and Nuclear Norm Regularization

Johns Hopkins University（约翰斯·霍普金斯大学）

We give a formal and complete characterization of the explicit regularizer induced by dropout in deep linear networks with squared loss. We show that (a) the explicit regularizer is composed of an $\ell_2$-path regularizer and other terms that are also re-scaling invariant, (b) the convex envelope of the induced regularizer is the squared nuclear norm of the network map, and (c) for a sufficiently large dropout rate, we characterize the global optima of the dropout objective. We validate our theoretical findings with empirical results.

2019-05-28cs.AIcs.LGstat.ML

增强型逆混合加权：确定性与随机分布偏移下的鲁棒推断

Augmented Inverse Hybrid Weighting: Robust Inference under Deterministic and Random Distribution Shifts

该研究针对分布偏移问题，提出AIHW方法，通过分离系统偏移与残余随机扰动并权衡两类不确定性，在多站点数据集上验证其可降低均方误差、提升覆盖范围，具有良好鲁棒性。

2026-08-04cs.LGmath.STstat.ME

Logitron: Perceptron-augmented classification model based on an extended logistic loss function

School of Liberal Arts, Korea University of Technology and Education（韩国技术教育大学文理学院）

Classification is the most important process in data analysis. However, due to the inherent non-convex and non-smooth structure of the zero-one loss function of the classification model, various convex surrogate loss functions such as hinge loss, squared hinge loss, logistic loss, and exponential loss are introduced. These loss functions have been used for decades in diverse classification models, such as SVM (support vector machine) with hinge loss, logistic regression with logistic loss, and Adaboost with exponential loss and so on. In this work, we present a Perceptron-augmented convex clas…

2019-04-05cs.ITcs.LGmath.IT