首页/论文/2003.03687

arXiv 2003.036872020-03cs.LGstat.ML

Some Geometrical and Topological Properties of DNNs' Decision Boundaries

Beijing University of Technology（北京工业大学）

Bo Liu, Mengya Shen

英文摘要：

Geometry and topology of decision regions are closely related with classification performance and robustness against adversarial attacks. In this paper, we use differential geometry to theoretically explore the geometrical and topological properties of decision regions produced by deep neural networks (DNNs). The goal is to obtain some geometrical and topological properties of decision boundaries for given DNN models, and provide some principled guidance to design and regularization of DNNs. First, we present the curvatures of decision boundaries in terms of network parameters, and give sufficient conditions on network parameters for producing flat or developable decision boundaries. Based on the Gauss-Bonnet-Chern theorem in differential geometry, we then propose a method to compute the Euler characteristics of compact decision boundaries, and verify it with experiments.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

简单是否胜过复杂？比较深度神经网络对抗鲁棒性的策略

Does simple trump complex? Comparing strategies for adversarial robustness in DNNs

Faculty of Engineering, North-West University, South Africa（南非北开普大学工程学院）；Centre for Artificial Intelligence Research, South Africa（南非人工智能研究中心）；National Institute for Theoretical and Computational Sciences, South Africa（南非理论与计算科学国家研究所）

本研究聚焦DNNs对抗鲁棒性，通过输入空间边界视角分离评估两种最大化边界的对抗训练技术组件（简单损失函数修改法与复杂动态感知鲁棒训练法），基于VGG-16在CIFAR-10上的实验揭示关键有效元素，为鲁棒DNN设计提供见解。

2025-08-25cs.LG

基于集成边界近似的对抗检测

Adversarial Detection by Approximation of Ensemble Boundary

University of Surrey（萨里大学）

针对深度神经网络易受对抗攻击的问题，提出基于Walsh系数的二分类DNN集成模型对抗检测方法，通过近似决策边界、利用曲率变化差异检测攻击，还可辅助理解DNN特性，方法具备跨领域应用潜力。

2025-01-10cs.AIcs.CRcs.CV

从图像边界对深度神经网络进行不可感知的对抗性攻击

Imperceptible Adversarial Attack on Deep Neural Networks from Image Boundary

University of Nebraska Omaha（内布拉斯加大学奥马哈分校）

本研究提出一种从图像边界进行不可感知的对抗性攻击方法，仅用32%边界内容即可高效攻击多种DNN模型，平均成功率95.2%，并揭示边界对模型行为的主导作用。

2023-08-29cs.CRcs.CVcs.LG

灾难性过拟合的层感知分析：揭示伪鲁棒捷径依赖

Layer-Aware Analysis of Catastrophic Overfitting: Revealing the Pseudo-Robust Shortcut Dependency

Sydeny AI Centre（悉尼人工智能中心）；School of Computer Science, The University of Sydney（悉尼大学计算机科学学院）；Hong Kong Baptist University（香港浸会大学）；BNRist Center, Tsinghua University（清华大学BNRist中心）

该研究针对单步对抗训练中的灾难性过拟合问题，通过层感知分析发现前层伪鲁棒捷径依赖是其诱因，并提出层感知对抗权重扰动方法，有效防止灾难性过拟合并提升模型鲁棒性。

2024-09-14cs.LG

通过学习受限对抗多面体（CAP）增强对抗攻击鲁棒性

Robustness Against Adversarial Attacks via Learning Confined Adversarial Polytopes

University of Waterloo（滑铁卢大学）

本文提出通过学习受限对抗多面体（CAP）约束范数有界扰动下的可达输出，使对抗多面体避开决策边界，从而提升 DNN 对 AutoAttack 等攻击的鲁棒性。

2024-01-20cs.CRcs.LG

Laplacian Networks: Bounding Indicator Function Smoothness for Neural Network Robustness

IMT Atlantique（IMT大西洋学院）；University of Southern California（南加州大学）

For the past few years, Deep Neural Network (DNN) robustness has become a question of paramount importance. As a matter of fact, in sensitive settings misclassification can lead to dramatic consequences. Such misclassifications are likely to occur when facing adversarial attacks, hardware failures or limitations, and imperfect signal acquisition. To address this question, authors have proposed different approaches aiming at increasing the robustness of DNNs, such as adding regularizers or training using noisy examples. In this paper we propose a new regularizer built upon the Laplacian of simi…

2018-10-02cs.CRcs.LGstat.ML

更多相关论文收起更多论文

最优传输诱导的对抗性样本对抗分布外过度自信

Optimal Transport-Induced Samples against Out-of-Distribution Overconfidence

Guangzhou University（广州大学）；SmartMore Corporation（SmartMore公司）；University of Science and Technology of China（中国科学技术大学）；Northwestern Polytechnical University（西北工业大学）；Guangdong Key Laboratory of Industrial Control System Security（广东工业控制系统安全重点实验室）

本文提出通过最优传输诱导的边界生成分布外样本，以缓解模型在不确定区域的过度自信预测问题。

2026-01-29cs.CVcs.LG

通过校准嵌入距离诊断基于AI的漏洞检测中的泛化边界

Diagnosing Generalization Boundaries in AI-Based Vulnerability Detection via Calibrated Embedding Distances

University of Koblenz（科布伦茨大学）；Fraunhofer ISST（弗劳恩霍夫ISST研究所）

该研究针对CodeBERT与Qwen2.5-Coder，利用校准嵌入距离揭示AI漏洞检测在分布偏移下的泛化边界，发现跨项目偏移退化最剧烈，误报与几何 proximity 相关，无需辅助模型即可诊断模型退化。

2026-10-14cs.LGcs.SE

SyReNN: A Tool for Analyzing Deep Neural Networks

University of California, Davis（加州大学戴维斯分校）

Deep Neural Networks (DNNs) are rapidly gaining popularity in a variety of important domains. Formally, DNNs are complicated vector-valued functions which come in a variety of sizes and applications. Unfortunately, modern DNNs have been shown to be vulnerable to a variety of attacks and buggy behavior. This has motivated recent work in formally analyzing the properties of such DNNs. This paper introduces SyReNN, a tool for understanding and analyzing a DNN by computing its symbolic representation. The key insight is to decompose the DNN into linear functions. Our tool is designed for analyses …

2021-01-09cs.LGcs.PL

Empirical Studies on the Properties of Linear Regions in Deep Neural Networks

Huazhong University of Science and Technology（华中科技大学）

A deep neural network (DNN) with piecewise linear activations can partition the input space into numerous small linear regions, where different linear functions are fitted. It is believed that the number of these regions represents the expressivity of the DNN. This paper provides a novel and meticulous perspective to look into DNNs: Instead of just counting the number of the linear regions, we study their local properties, such as the inspheres, the directions of the corresponding hyperplanes, the decision boundaries, and the relevance of the surrounding regions. We empirically observed that d…

2020-04-28cs.LGstat.ML

深度学习中奥卡姆剃刀的几何建模

A Geometric Modeling of Occam's Razor in Deep Learning

CSIRO’s Data61（澳大利亚CSIRO数据61研究所）；Sony Computer Science Laboratories Inc. (Sony CSL)（索尼计算机科学实验室有限公司）

本文用奇异半 Riemann 几何与信息论刻画 DNN 参数空间，以 Fisher 信息矩阵显著维度定义局部维度并推导复杂性度量，解释高参数 DNN 的良好泛化。

2025-06-11cs.LGstat.ML

A Linear Algebraic Approach to Model Parallelism in Deep Learning

Virginia Tech（弗吉尼亚理工大学）

Training deep neural networks (DNNs) in large-cluster computing environments is increasingly necessary, as networks grow in size and complexity. Local memory and processing limitations require robust data and model parallelism for crossing compute node boundaries. We propose a linear-algebraic approach to model parallelism in deep learning, which allows parallel distribution of any tensor in the DNN. Rather than rely on automatic differentiation tools, which do not universally support distributed memory parallelism models, we show that parallel data movement operations, e.g., broadcast, sum-re…

2020-06-04cs.DCcs.LGstat.ML

设计中的低秩学习：网络架构与激活线性在梯度秩坍缩中的作用

Low-Rank Learning by Design: the Role of Network Architecture and Activation Linearity in Gradient Rank Collapse

Translational Research Center for Neuroimaging and Data Science (TReNDs)（神经成像与数据科学转化研究中心（TReNDs））；Georgia State University（佐治亚州立大学）；Georgia Institute of Technology（佐治亚理工学院）；Emory University（埃默里大学）；Maynooth University（梅努斯大学）

本文全面研究了DNNs的梯度秩，理论分析了架构选择和数据结构对梯度秩界限的影响，并揭示激活线性、瓶颈层等设计如何影响低秩学习，为深度学习设计提供指导。

2024-02-09cs.LG

结合注意力与卷积的参数化编码缓解神经偏微分方程求解器的谱偏置

Parametric Encoding with Attention and Convolution Mitigate Spectral Bias of Neural Partial Differential Equation Solvers

University of California, Irvine（加州大学欧文分校）

针对深度神经网络求解偏微分方程时随复杂度增加而精度下降及存在谱偏置的问题，提出参数化网格卷积注意力网络（PGCAN），通过网格编码器与注意力解码器结合，在无标签数据下有效缓解谱偏置并提升求解精度。

2024-03-22cs.LG

On the Compressive Power of Deep Rectifier Networks for High Resolution Representation of Class Boundaries

The University of Western Australia（西澳大利亚大学）

This paper provides a theoretical justification of the superior classification performance of deep rectifier networks over shallow rectifier networks from the geometrical perspective of piecewise linear (PWL) classifier boundaries. We show that, for a given threshold on the approximation error, the required number of boundary facets to approximate a general smooth boundary grows exponentially with the dimension of the data, and thus the number of boundary facets, referred to as boundary resolution, of a PWL classifier is an important quality measure that can be used to estimate a lower bound o…

2017-08-24cs.AIcs.LGstat.ML

包含符号层的深度神经网络中的符号正确性

Symbol Correctness in Deep Neural Networks Containing Symbolic Layers

School of Computing and Information Systems, University of Melbourne（墨尔本大学计算与信息系统学院）

本文提出符号正确性概念，作为神经符号深度神经网络（NS-DNN）中神经层预测符号与真实符号一致性的原则，证明其对可解释性和迁移学习必要，并揭示训练权衡。

2024-02-06cs.AIcs.LG

超越分类器的模式连通性：来自生成式与对比式模型的证据

Mode Connectivity Beyond Classifiers: Evidence from Generative and Contrastive Models

University of Electronic Science and Technology of China（电子科技大学）；Delft University of Technology（代尔夫特理工大学）

本文将模式连通性研究拓展至生成式与对比式模型领域，针对DDPM与CLIP架构提出连接算法，首次证实独立训练的DDPM与NanoCLIP模式间存在模式连通性，为理解现代模型损失景观提供新视角。

2026-09-01cs.LG