Profile Entropy: A Fundamental Measure for the Learnability and Compressibility of Discrete Distributions

神经流形中的信息挫折：香农瓶颈与可学习性的极限

Informational Frustration in Neural Manifolds: Shannon Bottlenecks and the Limits of Learnability

提出熵可学习性视界（ELH）和香农-拓扑瓶颈定理，揭示数据流形熵、决策边界拓扑熵与权重空间冯·诺依曼熵之间的平衡关系，解释过参数化网络泛化现象并导出熵梯度下降（EGD）算法。

Learnability, Sample Complexity, and Hypothesis Class Complexity for Regression Models

The goal of a learning algorithm is to receive a training data set as input and provide a hypothesis that can generalize to all possible data points from a domain set. The hypothesis is chosen from hypothesis classes with potentially different complexities. Linear regression modeling is an important category of learning algorithms. The practical uncertainty of the target samples affects the generalization performance of the learned model. Failing to choose a proper model or hypothesis class can lead to serious issues such as underfitting or overfitting. These issues have been addressed by alte…

平均场贝叶斯神经网络中的宽度鲁棒可学习性

Width-Robust Learnability in Mean-Field Bayesian Neural Networks

研究平均场贝叶斯神经网络中无限宽度与多项式宽度下布尔立方体目标的可学习性问题，通过约化熵建立联系，利用子采样证明宽度鲁棒可学习性定理，表明无限宽度平均场极限能清晰描述学习，不引入虚假泛化能力。

噪声或对抗性扰动下样本可压缩分布的鲁棒可学习性

Robust Learnability of Sample-Compressible Distributions under Noisy or Adversarial Perturbations

本文研究受噪声或对抗性损坏样本下的分布学习，提出扰动量化框架，证明样本可压缩分布族在充要条件下仍可PAC学习，并给出高维均匀混合与高斯混合的新样本复杂度界。

Scalable Model Compression by Entropy Penalized Reparameterization

We describe a simple and general neural network weight compression approach, in which the network parameters (weights and biases) are represented in a "latent" space, amounting to a reparameterization. This space is equipped with a learned probability model, which is used to impose an entropy penalty on the parameter representation during training, and to compress the representation using a simple arithmetic coder after training. Classification accuracy and model compressibility is maximized jointly, with the bitrate--accuracy trade-off specified by a hyperparameter. We evaluate the method on …

嵌入多边形符号传递熵 (EPSTE)：一种用于神经影像时间序列传递熵估计的几何令牌与深度学习方法

Embedded Polygon Symbolic Transfer Entropy (EPSTE): A Geometric Token and Deep Learning Approach to Estimating Transfer Entropy in Neuroimaging Time Series

提出EPSTE框架，将传递熵估计转化为基于局部时间形态几何符号表示的可学习问题，结合循环神经网络与注意力多实例学习，在MEG数据上实现稳定有向依赖估计。