arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2502.02085cs.DScs.LG
收藏

一种新的基于拒绝采样的 $k$-$\mathtt{means}$++ 方法及其改进的权衡

A New Rejection Sampling Approach to $k$-$\mathtt{means}$++ With Improved Trade-Offs

Indian Institute of Technology Delhi(印度理工学院德里分校)

Poojan Shah, Shashwat Agrawal, Ragesh Jaiswal

AI总结:

本文提出基于拒绝采样的新方法加速 $k$-means++,在保持 $O(\log k)$ 期望竞争力的同时显著降低计算代价,并给出改进的代价与质量权衡,通过实验验证了其有效性。

摘要:

$k$-means++ 种子点算法(Arthur & Vassilvitskii, 2007)在实践中被广泛用于 $k$-means 聚类问题,其目标是将数据集 $\mathcal{X} \subset \mathbb{R}^d$ 划分为 $k$ 个簇。该算法的流行源于其简单性以及可证明的保证:在期望上与最优解相比具有 $O(\log k)$ 的竞争力。然而,其运行时间为 $O(|\mathcal{X}|kd)$,这使得它在大型数据集上代价高昂。在这项工作中,我们提出了一种简单有效的基于拒绝采样的方法来加速 $k$-means++。我们的第一种方法运行时间为 $\tilde{O}(\mathtt{nnz}(\mathcal{X}) + \beta k^2d)$,同时在期望上仍保持 $O(\log k)$ 的竞争力。这里,$\beta$ 是一个参数,表示数据集方差与期望中最优 $k$-means 代价的比值,$\tilde{O}$ 隐藏了关于 $k$ 和 $|\mathcal{X}|$ 的对数因子。我们的第二种方法在计算代价与解质量之间给出了新的权衡。除了 $k$-means++ 的 $O(\log k)$ 保证之外,它额外引入一个尺度不变因子 $k^{-\Omega(m/\beta)} \operatorname{Var}(\mathcal{X})$,改进了 (Bachem et al, 2016a) 的结果,后者在运行时间仍为 $\tilde{O}(\mathtt{nnz}(\mathcal{X}) + mk^2d)$ 的情况下会额外引入因子 $m^{-1}\operatorname{Var}(\mathcal{X})$。我们进行了大量的实证评估,以验证我们的理论结果,并展示我们的方法在真实数据集上的有效性。

英文摘要:

The $k$-$\mathtt{means}$++ seeding algorithm (Arthur & Vassilvitskii, 2007) is widely used in practice for the $k$-means clustering problem where the goal is to cluster a dataset $\mathcal{X} \subset \mathbb{R} ^d$ into $k$ clusters. The popularity of this algorithm is due to its simplicity and provable guarantee of being $O(\log k)$ competitive with the optimal solution in expectation. However, its running time is $O(|\mathcal{X}|kd)$, making it expensive for large datasets. In this work, we present a simple and effective rejection sampling based approach for speeding up $k$-$\mathtt{means}$++. Our first method runs in time $\tilde{O}(\mathtt{nnz} (\mathcal{X}) + βk^2d)$ while still being $O(\log k )$ competitive in expectation. Here, $β$ is a parameter which is the ratio of the variance of the dataset to the optimal $k$-$\mathtt{means}$ cost in expectation and $\tilde{O}$ hides logarithmic factors in $k$ and $|\mathcal{X}|$. Our second method presents a new trade-off between computational cost and solution quality. It incurs an additional scale-invariant factor of $ k^{-Ω( m/β)} \operatorname{Var} (\mathcal{X})$ in addition to the $O(\log k)$ guarantee of $k$-$\mathtt{means}$++ improving upon a result of (Bachem et al, 2016a) who get an additional factor of $m^{-1}\operatorname{Var}(\mathcal{X})$ while still running in time $\tilde{O}(\mathtt{nnz}(\mathcal{X}) + mk^2d)$. We perform extensive empirical evaluations to validate our theoretical results and to show the effectiveness of our approach on real datasets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑