信息空间的草图、嵌入和抽象性简化
Sketching, Embedding, and Dimensionality Reduction for Information Spaces
摘要：
像Hellinger距离和Jensen-Shannon分歧这样的信息距离在信息论和机器学习中有着深刻的根源。它们广泛用于数据分析，特别是当比较的对象是从数据构建的高维经验概率分布时。然而，我们缺乏在应用程序中有效地使用信息距离所需的通用工具，并且具有任何类型的可证明保证。我们不能轻易地勾画出这些距离，或者将它们嵌入到行为更好的空间中，甚至在保持数据概率结构的同时降低空间的维数。在本文中，我们建立这些工具的信息距离-无论是海林格距离和詹森-香农分歧，以及相关的措施，如$χ^2$分歧。我们首先表明，他们可以有效地勾勒（即乘法误差在次线性空间）的聚合流模型。这个结果是指数强于已知的上限，在严格的旋转栅门流模型中绘制这些距离。第二，我们展示了有限维嵌入的结果为詹森-香农和$χ^2$分歧，保持对明智的距离。最后，我们证明了降维结果的Hellinger，詹森-香农，和$χ^2$分歧，保留信息几何的分布（特别是，通过保留单纯形结构的空间）。虽然我们上面的第二个结果已经暗示了这些分歧可以显式地嵌入到欧几里得空间中，但保留单纯形结构是重要的，因为它允许我们继续在约化空间中进行推理。本质上，我们不仅保留了距离结构，而且保留了空间的基本几何结构。
英文摘要：
Information distances like the Hellinger distance and the Jensen-Shannon divergence have deep roots in information theory and machine learning. They are used extensively in data analysis especially when the objects being compared are high dimensional empirical probability distributions built from data. However, we lack common tools needed to actually use information distances in applications efficiently and at scale with any kind of provable guarantees. We can't sketch these distances easily, or embed them in better behaved spaces, or even reduce the dimensionality of the space while maintaining the probability structure of the data. In this paper, we build these tools for information distances---both for the Hellinger distance and Jensen--Shannon divergence, as well as related measures, like the $χ^2$ divergence. We first show that they can be sketched efficiently (i.e. up to multiplicative error in sublinear space) in the aggregate streaming model. This result is exponentially stronger than known upper bounds for sketching these distances in the strict turnstile streaming model. Second, we show a finite dimensionality embedding result for the Jensen-Shannon and $χ^2$ divergences that preserves pair wise distances. Finally we prove a dimensionality reduction result for the Hellinger, Jensen--Shannon, and $χ^2$ divergences that preserves the information geometry of the distributions (specifically, by retaining the simplex structure of the space). While our second result above already implies that these divergences can be explicitly embedded in Euclidean space, retaining the simplex structure is important because it allows us to continue doing inference in the reduced space. In essence, we preserve not just the distance structure but the underlying geometry of the space.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
