从动力学到决策：深度分类器中Lyapunov谱与决策边界的数学理论

中文摘要 AI 辅助

深度分类器的定义不仅取决于其产生的决策，还取决于形成该决策的变换序列。将这种演化视为跨层的动力系统，为探究决策几何如何通过深度涌现以及边界动力学特征可追溯多远提供了自然框架。我们将前馈分类器建模为有限、非自治的离散动力系统，其中各层扮演离散时间步的角色。我们研究了数据样本在分类器深度中动力学轨迹的有限时间最大Lyapunov指数（FTMLE）。FTMLE衡量相邻轨迹的收敛/发散速率。我们将观测端点从概率向后移动到logits，再移动到隐藏表示。对于高斯类，我们证明了概率层面的FTMLE携带清晰的决策边界几何特征，其主导方向与边界法线对齐。向后移动一步到logits，我们证明这种关系不再普遍成立，而是关键取决于分类器的训练方式，特别是损失函数的选择。进一步向后移动到隐藏表示，这种联系变得更加有条件：与边界相关的FTMLE可以持续存在，但仅在可识别的结构条件下。我们提出了几何感知微调来重构分类器的隐藏FTMLE，并提出了高隐藏FTMLE在决策边界附近保证集中的条件。通过数值结果，我们展示了理论结果的普遍性和有效性。理解数据样本在分类器层间演化的过程，为识别边界相关敏感性出现的位置以及开发层感知正则化策略提供了原则性基础。

英文摘要

A deep classifier is defined not only by the decision it produces, but also by the sequence of transformations through which that decision is formed. Treating this evolution as a dynamical system across layers provides a natural framework for asking how decision geometry emerges through depth and how far back we can trace a boundary's dynamical signature. We model a feed-forward classifier as a finite, nonautonomous discrete dynamical system, with layers playing the role of discrete time steps. We study the Finite-Time Maximum Lyapunov Exponent (FTMLE) of the data samples' dynamical trajectory through depths of the classifier. The FTMLE measures the rate of convergence/divergence of nearby trajectories. We move the observation endpoint backward from probabilities to logits and then to hidden representations. For Gaussian classes, we prove that probability-level FTMLE carries a clear geometric signature of the decision boundary, with its dominant direction aligned with the boundary normal. Moving one step backward to the logits, we prove this relationship is no longer universal but depends critically on how the classifier is trained, particularly on the choice of loss function. Moving further backward to the hidden representation, the connection becomes more conditional: boundary-related FTMLE can persist, but only under identifiable structural conditions. We propose geometry-aware fine-tuning for restructuring the classifier's hidden FTMLE, and propose conditions for guaranteed concentration of high hidden FTMLE near the decision boundary. Through our numerical results, we show the generality and validity of our theoretical results. Understanding the evolution of data samples as traveling through the layers of classifier provides a principled foundation for identifying where boundary-relevant sensitivity emerges and for developing layer-aware regularization strategies.

发表机构

• Colorado State University（科罗拉多州立大学）

机构由 AI 辅助整理，请以论文原文为准。