首页/论文/2011.14211

arXiv 2011.142112020-11-28cs.LGcs.CVstat.ML

Curvature Regularization to Prevent Distortion in Graph Embedding

Jilin University（吉林大学）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）

Hongbin Pei, Bingzhe Wei, Kevin Chen-Chuan Chang, Chunxu Zhang, Bo Yang

备注：: Published as a conference paper at NeurIPS 2020

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

英文摘要相关论文

英文摘要：

Recent research on graph embedding has achieved success in various applications. Most graph embedding methods preserve the proximity in a graph into a manifold in an embedding space. We argue an important but neglected problem about this proximity-preserving strategy: Graph topology patterns, while preserved well into an embedding manifold by preserving proximity, may distort in the ambient embedding Euclidean space, and hence to detect them becomes difficult for machine learning models. To address the problem, we propose curvature regularization, to enforce flatness for embedding manifolds, thereby preventing the distortion. We present a novel angle-based sectional curvature, termed ABS curvature, and accordingly three kinds of curvature regularization to induce flat embedding manifolds during graph embedding. We integrate curvature regularization into five popular proximity-preserving embedding methods, and empirical results in two applications show significant improvements on a wide range of open graph datasets.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

Early-Learning Regularization Prevents Memorization of Noisy Labels

New York University（纽约大学）；NYU School of Medicine（纽约大学医学院）

We propose a novel framework to perform classification via deep learning in the presence of noisy annotations. When trained on noisy labels, deep neural networks have been observed to first fit the training data with clean labels during an "early learning" phase, before eventually memorizing the examples with false labels. We prove that early learning and memorization are fundamental phenomena in high-dimensional classification tasks, even in simple linear models, and give a theoretical explanation in this setting. Motivated by these findings, we develop a new technique for noisy classificatio…

2020-10-22cs.CVcs.LGstat.ML

Neuron-Specific Dropout: A Deterministic Regularization Technique to Prevent Neural Networks from Overfitting & Reduce Dependence on Large Training Samples

Perry High School（佩里高中）

In order to develop complex relationships between their inputs and outputs, deep neural networks train and adjust large number of parameters. To make these networks work at high accuracy, vast amounts of data are needed. Sometimes, however, the quantity of data needed is not present or obtainable for training. Neuron-specific dropout (NSDropout) is a tool to address this problem. NSDropout looks at both the training pass, and validation pass, of a layer in a model. By comparing the average values produced by each neuron for each class in a data set, the network is able to drop targeted units. …

2022-01-13cs.LGstat.ML

超越正则化有限混合：理论保证与实际限制

Transcendental Regularization of Finite Mixtures:Theoretical Guarantees and Practical Limitations

Rochester Institute of Technology（罗切斯特技术学院）

本文提出超越正则化方法，用于解决有限混合模型在高维空间中的退化问题，提供理论保证并揭示其实际限制。

2026-02-03cs.LGstat.ML

抗离群点神经网络训练：变分正则化与截尾损失相结合以防止功能崩溃

Outlier-robust neural network training: variation regularization meets trimmed loss to prevent functional breakdown

Institute of Statistical Mathematics（统计数学研究所）；The Graduate University for Advanced Studies, SOKENDAI（高等研究大学、SOKENDAI）；RIKEN（理化学研究所）；Joint Support-Center for Data Science Research（数据科学联合支持中心）

本文提出结合变换截尾损失TTL与高阶变分正则化HOVR的神经网络训练框架，通过控制模型容量防止拟合离群点导致功能崩溃，并给出高崩溃点与随机优化算法收敛性保证。

2025-05-13cs.AIcs.LGstat.CO

Rate-Regularization and Generalization in VAEs

Northeastern University（东北大学）；Boston College（波士顿学院）

Variational autoencoders optimize an objective that combines a reconstruction loss (the distortion) and a KL term (the rate). The rate is an upper bound on the mutual information, which is often interpreted as a regularizer that controls the degree of compression. We here examine whether inclusion of the rate also acts as an inductive bias that improves generalization. We perform rate-distortion analyses that control the strength of the rate term, the network capacity, and the difficulty of the generalization problem. Decreasing the strength of the rate paradoxically improves generalization in…

2021-03-25cs.LGstat.ML

硬正则化防止无数据增强的深度在线聚类崩溃

Hard Regularization to Prevent Deep Online Clustering Collapse without Data Augmentation

University of Edinburgh（爱丁堡大学）；Vienna University of Technology（维也纳工业大学）；University of Oxford（牛津大学）

提出一种基于贝叶斯框架的硬分配正则化方法，无需数据增强即可防止在线深度聚类崩溃，在多个数据集上表现更稳健且聚类更准确。

2024-03-13cs.LG

更多相关论文收起更多论文

$λ$-JEPA：用于自监督学习的谱抗坍缩正则化

$λ$-JEPA Spectral Anti-Collapse Regularization for Self-Supervised Learning

Institute of Science and Technology Austria (ISTA)（奥地利科学技术研究所）

提出SACReg谱抗坍缩正则化器，基于λ-平衡分析，应用于JEPA得到λ-JEPA，在图像和视频自监督学习中提升表征秩和下游迁移性能。

2026-09-30cs.CVcs.LG

通过异常对抗样本正则化消除灾难性过拟合

Eliminating Catastrophic Overfitting Via Abnormal Adversarial Examples Regularization

Sydney AI Centre, The University of Sydney（悉尼大学悉尼人工智能中心）

针对单步对抗训练中存在的灾难性过拟合问题，本研究发现异常对抗样本与分类器扭曲密切相关，提出异常对抗样本正则化方法，可在低计算开销下有效消除灾难性过拟合并提升对抗鲁棒性。

2024-09-14cs.LG

Avoiding unwanted results in locally linear embedding: A new understanding of regularization

National Sun Yat-sen University（国立中山大学）

We demonstrate that locally linear embedding (LLE) inherently admits some unwanted results when no regularization is used, even for cases in which regularization is not supposed to be needed in the original algorithm. The existence of one special type of result, which we call ``projection pattern'', is mathematically proved in the situation that an exact local linear relation is achieved in each neighborhood of the data. These special patterns as well as some other bizarre results that may occur in more general situations are shown by numerical examples on the Swiss roll with a hole embedded i…

2021-08-28cs.LGcs.NAmath.NA

用于公平自监督学习的无偏开放世界正则化

Unbiased Open World Regularization for Fair Self-Supervised Learning

研究自监督学习模型易产生虚假偏差问题，提出无偏开放世界正则化框架UOWReg，从全局目标转向条件目标保证表示与属性统计独立，经实验验证能减轻偏差、保持精度，还在新任务中防止子群体崩溃。

2026-07-27cs.LG

基于图的金融欺诈检测：校准风险评分与结构正则化

Graph-Based Financial Fraud Detection with Calibrated Risk Scoring and Structural Regularization

Brandeis University（布雷纳大学）；University of California, Los Angeles（加州大学洛杉矶分校）；Georgetown University（乔治城大学）；Columbia University（哥伦比亚大学）；Stevens Institute of Technology（史蒂文斯理工学院）

本文提出一种图神经网络表示学习与风险鉴别框架，通过整合交易记录和身份信息构建交易图，利用多层消息传递机制和结构正则化提升欺诈检测的准确性和稳定性。

2026-05-12cs.LG

Partial Graph Reasoning for Neural Network Regularization

University of Sydney（悉尼大学）；University of New South Wales（新南威尔士大学）；Paige AI；Tencent AI Lab（腾讯人工智能实验室）

Regularizers help deep neural networks prevent feature co-adaptations. Dropout, as a commonly used regularization technique, stochastically disables neuron activations during network optimization. However, such complete feature disposal can affect the feature representation and network understanding. Toward better descriptions of latent representations, we present DropGraph that learns a regularization function by constructing a stand-alone graph from the backbone features. DropGraph first samples stochastic spatial feature vectors and then incorporates graph reasoning methods to generate feat…

2022-01-24cs.CVcs.LG

大规模稀疏特征嵌入模型的自适应正则化

Adaptive Regularization for Large-Scale Sparse Feature Embedding Models

Institute of Intelligent Technology（智能技术研究所）；Alibaba International Digital Commerce Group（阿里巴巴国际数字商业集团）

本文提出了一种自适应正则化方法，用于解决大规模稀疏特征嵌入模型中多 epoch 训练导致的过拟合问题，通过约束嵌入层范数预算提升模型性能。

2026-01-27cs.LGstat.ML

Optimizing Generative Adversarial Networks for Image Super Resolution via Latent Space Regularization

Natural images can be regarded as residing in a manifold that is embedded in a higher dimensional Euclidean space. Generative Adversarial Networks (GANs) try to learn the distribution of the real images in the manifold to generate samples that look real. But the results of existing methods still exhibit many unpleasant artifacts and distortions even for the cases where the desired ground truth target images are available for supervised learning such as in single image super resolution (SISR). We probe for ways to alleviate these problems for supervised GANs in this paper. We explicitly apply t…

2021-01-09cs.CVcs.LGeess.IV

超图p-拉普拉斯正则化在点云数据插值中的应用

Hypergraph $p$-Laplacian regularization on point clouds for data interpolation

China Jiliang University（中国计量大学）；Deutsches Elektronen-Synchrotron DESY（德国电子同步加速器研究所）；Universität Hamburg（汉堡大学）

本文提出基于超图的p-拉普拉斯正则化方法，用于提升点云数据插值中防止标记点尖峰发展的性能。

2025-03-17cs.LGcs.NAmath.AP

对数空间中的Guardrails：用于LLM对齐的安全令牌正则化

Guardrails in Logit Space: Safety Token Regularization for LLM Alignment

Applied Artificial Intelligence Initiative (A2I2)（应用人工智能倡议（A2I2））；Deakin University（德克萨斯大学）

本文提出安全令牌正则化（STR）方法，通过约束关键令牌的logits来保持细调模型的安全性，实验显示其在安全性和任务性能方面表现优异。

2026-04-19cs.LG

通过隐式正则化在音频一致性自编码器中学习线性性

Learning Linearity in Audio Consistency Autoencoders via Implicit Regularization

LTCI, Telecom Paris, Institut Polytechnique de Paris（LTCI，电信巴黎，巴黎高等理工学院）；Deezer Research（Deezer研究）

通过隐式正则化在音频一致性自编码器中诱导线性性，实现更直观高效的音频处理

2026-01-27cs.AIcs.LGcs.SD