发表机构
School of Computer Science; Shanghai Jiao Tong University（计算机学院; 上海交通大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文针对两类对象嵌入流形，提出并证明了随机可分性定理，通过投影测度集中分析技术，揭示了对象嵌入流形的几何与统计特性，为深度网络表示学习提供新机制。
AI 中文摘要
神经生物学研究与表示学习已发现，高维神经空间中同一类别对象的表示呈现出低维对象流形的特性，且不同对象流形在该神经空间中是线性可分的，但这些实验观测到的现象迄今缺乏严格的理论验证。本文针对两个不同类别的对象的嵌入流形，提出了一种新的随机可分性定理。首先，我们在一般条件下建立了嵌入流形的投影测度集中定理，开发了一种新的两层测度集中分析技术，该技术通过全期望法则统一两个估计界，以推导测度集中不等式。基于该测度集中定理，我们进一步证明了两个不同类别的对象的嵌入流形的随机可分性定理：若两个数据集具有不同的均值和有界的总方差，且投影方向满足非奇异性条件，则它们的样本以高概率成为线性可分的。本文的主要贡献有两点：1. 利用双律师尾界不等式，证明了高维空间中嵌入流形的投影集中特性；2. 确定了嵌入流形之间随机可分性的非奇异性条件，并严格证明了随机投影可分性定理。该定理不仅揭示了对象嵌入流形的几何与统计特性，还为深度网络中的表示学习提供了一种新机制。
英文摘要
Neurobiological studies and representation learning have observed that representations of objects belonging to the same category in high-dimensional neural spaces exhibit low-dimensional object manifold characteristics, and different object manifolds are linearly separable in these neural spaces. However, these experimentally observed phenomena lack rigorous theoretical validation to date. This paper proposes a new stochastic separability theorem for embedding manifolds of two different object categories. First, we establish a projection measure concentration theorem for embedding manifolds under general conditions. We develop a new two-layer measure concentration analysis technique, which unifies two estimation bounds via the law of total expectation to derive measure concentration inequalities. Based on the measure concentration theorem, we further prove a stochastic separability theorem for embedding manifolds of two different object categories. If two datasets have distinct means and bounded total variances, their samples become linearly separable with high probability, provided that the projection direction satisfies a non-singularity condition. The main contributions of this paper are twofold: 1. We prove the projection concentration properties of embedding manifolds in high-dimensional spaces by using two-lawyer tail-bound inequalities. 2. We identify a non-singularity condition for the stochastic separability between embedding manifolds, and rigorously prove the stochastic projection separability theorem. The theorem not only uncovers geometric and statistical properties of the object embedding manifolds, but also provides a novel mechanism for representation learning in deep networks.
