arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2206.07554cs.DS
收藏

图流中的层次聚类:单遍算法和空间下界

Hierarchical Clustering in Graph Streams: Single-Pass Algorithms and Space Lower Bounds

Sepehr Assadi, Vaggos Chatziafratis, Jakub Łącki, Vahab Mirrokni, Chen Wang

摘要:

层次聚类(HC)问题包括建立一个层次的聚类来表示给定的数据集。受现代大规模应用的启发,我们研究了\streaming模型中的问题,在该模型中,内存严重受限,只允许对输入进行一次或很少的传递。具体来说,我们调查是否可以获得一个良好的层次聚类,或者至少我们是否可以近似估计的最佳层次的值。为了衡量质量的层次结构,我们使用的HC最小化目标介绍Dasgupta。假设输入是一个$n$ -顶点加权图,其边到达流中,我们得到以下关于空间与精度权衡的结果:* 使用$O(n\cdot \text{polylog}\,{n})$空间,我们开发了一个单遍算法,其近似比匹配当前最好的离线算法。* 当空间是有限的,即$n^{1-o(1)}$,我们证明,没有算法甚至可以估计最佳HC树的值内的$o(\frac{\log{n}}{\log\log{n}})$因素,即使允许$\text{polylog}{\,{n}}$通过输入。* 在最严格的设置$\text{polylog}\,{n}$空间,我们排除了算法,甚至可以区分“高度”与“差”聚类图,即,图形之间有一个$n^{1/2-o(1)}$因素差距HC目标值。* 最后,我们证明,任何单通流算法,计算一个最佳的HC树需要存储几乎整个输入,即使允许的指数时间。我们的算法结果建立了一个一般的结构性结果,证明了输入图的切割稀疏可以保持“平衡”HC树的成本在一个常数因子。我们的下界结果包括一个新的流下界的一个新的问题“一对多扩展器”,这可以是独立的利益。

英文摘要:

The Hierarchical Clustering (HC) problem consists of building a hierarchy of clusters to represent a given dataset. Motivated by the modern large-scale applications, we study the problem in the \streaming model, in which the memory is heavily limited and only a single or very few passes over the input are allowed. Specifically, we investigate whether a good hierarchical clustering can be obtained, or at least whether we can approximately estimate the value of the optimal hierarchy. To measure the quality of a hierarchy, we use the HC minimization objective introduced by Dasgupta. Assuming that the input is an $n$-vertex weighted graph whose edges arrive in a stream, we derive the following results on space-vs-accuracy tradeoffs: * With $O(n\cdot \text{polylog}\,{n})$ space, we develop a single-pass algorithm, whose approximation ratio matches the currently best offline algorithm. * When the space is more limited, namely, $n^{1-o(1)}$, we prove that no algorithm can even estimate the value of optimum HC tree to within an $o(\frac{\log{n}}{\log\log{n}})$ factor, even when allowed $\text{polylog}{\,{n}}$ passes over the input. * In the most stringent setting of $\text{polylog}\,{n}$ space, we rule out algorithms that can even distinguish between "highly"-vs-"poorly" clusterable graphs, namely, graphs that have an $n^{1/2-o(1)}$ factor gap between their HC objective value. * Finally, we prove that any single-pass streaming algorithm that computes an optimal HC tree requires to store almost the entire input even if allowed exponential time. Our algorithmic results establish a general structural result that proves that cut sparsifiers of input graph can preserve cost of "balanced" HC trees to within a constant factor. Our lower bound results include a new streaming lower bound for a novel problem "One-vs-Many-Expanders", which can be of independent interest.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑