用熵中的相变分析训练-第一部分：一般理论
Analyzing Training Using Phase Transitions in Entropy---Part I: General Theory
摘要：
我们分析相变中的条件熵的序列所造成的条件变量的变化。例如，当训练以学习系统的参数时，发生这种转变，因为从训练阶段到数据阶段的转变导致测量的系统响应的条件熵的不连续跳跃。对于大规模的系统，我们提出了一种方法，计算上的互信息得到一次训练的界限，并表明，这个界限可以计算使用的条件熵的两个导数之间的差异。系统模型不需要参数的高斯性或线性，并且不需要任何未知参数的最坏情况噪声近似或显式估计。该模型适用于通信、信号处理和机器学习中广泛的算法和方法，这些算法和方法将训练作为其操作的一部分。
英文摘要：
We analyze phase transitions in the conditional entropy of a sequence caused by a change in the conditional variables. Such transitions happen, for example, when training to learn the parameters of a system, since the transition from the training phase to the data phase causes a discontinuous jump in the conditional entropy of the measured system response. For large-scale systems, we present a method of computing a bound on the mutual information obtained with one-shot training, and show that this bound can be calculated using the difference between two derivatives of a conditional entropy. The system model does not require Gaussianity or linearity in the parameters, and does not require worst-case noise approximations or explicit estimation of any unknown parameters. The model applies to a broad range of algorithms and methods in communication, signal processing, and machine learning that employ training as part of their operation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
