基于指数凹性的无干扰信息集中
Dimension-free Information Concentration via Exp-Concavity
摘要:
概率测度的信息集中在学习理论中有重要的意义。最近,人们发现,一个对数凹分布的信息内容集中在他们的微分熵,虽然与环境尺寸的不愉快的依赖。在这项工作中,我们证明了如果对数凹分布的势是指数凹的,这是在线和统计学习中快速率的核心概念,那么信息的集中度可以进一步提高到仅取决于指数凹参数,因此,它可以是维度无关的。我们证明的核心是方差Brascamp-Lieb不等式的一个新颖而简单的应用。在学习理论的背景下,我们的信息集中结果立即意味着许多先前的边界仅在预期中保持的高概率结果。
英文摘要:
Information concentration of probability measures have important implications in learning theory. Recently, it is discovered that the information content of a log-concave distribution concentrates around their differential entropy, albeit with an unpleasant dependence on the ambient dimension. In this work, we prove that if the potentials of the log-concave distribution are exp-concave, which is a central notion for fast rates in online and statistical learning, then the concentration of information can be further improved to depend only on the exp-concavity parameter, and hence, it can be dimension independent. Central to our proof is a novel yet simple application of the variance Brascamp-Lieb inequality. In the context of learning theory, our concentration-of-information result immediately implies high-probability results to many of the previous bounds that only hold in expectation.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。