arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.04562stat.MEstat.AP
收藏

Spherical Double K-Means:一种用于文本数据分析的联合聚类方法

Spherical Double K-Means: a co-clustering approach for text data analysis

Ilaria Bombelli, Domenica Fioredistella Iezzi, Emiliano Seri, Maurizio Vichi

AI总结:

针对文本分析中需同时聚类文档与词项的需求,提出利用余弦相似度的Spherical Double K-Means方法,可提升主题识别与可解释性,经美国总统就职演说语料库验证能有效发掘文本潜在模式。

摘要:

在文本分析领域,Spherical K-means(SKM,球面K均值)是一种专门的k-means聚类算法,广泛用于对以高维稀疏词项-文档矩阵表示的文档进行聚类,这类矩阵通常采用TF-IDF等技术进行归一化。研究人员常常不仅需要对文档聚类,还需要将与之关联的词项聚为连贯的组。为满足这种双重聚类需求,我们提出了Spherical Double K-Means(SDKM,球面双K均值)——一种可同时对文档和词项进行聚类的全新方法。该方法具备多项优势:首先,通过整合文档与词项的聚类过程,SDKM能更深入地揭示内容与词汇之间的关系,从而实现更高效的主题识别和关键词提取。此外,这种双层聚类有助于理解文档簇中的宏观主题和特定术语,提升了结果的可解释性。SDKM利用余弦相似度有效应对文本数据固有的高维性和稀疏性,提升了计算效率。不仅如此,该方法能够捕捉主题内容随时间的动态变化,非常适合应用于快速发展的领域。最终,SDKM为推进文本挖掘工作提供了一个综合框架,有助于发掘对稳健数据分析至关重要的细微模式和结构。我们将SDKM应用于1789年乔治·华盛顿至2021年乔·拜登期间的美国总统就职演说语料库。分析结果揭示了与重要历史主题和时期相对应的 distinct 词簇和文档簇,展现了该方法助力深化数据理解的能力。我们的研究结果证明了SDKM在发掘文本数据潜在模式方面的有效性。

英文摘要:

In text analysis, Spherical K-means (SKM) is a specialized k-means clustering algorithm widely utilized for grouping documents represented in high-dimensional, sparse term-document matrices, often normalized using techniques like TF-IDF. Researchers frequently seek to cluster not only documents but also the terms associated with them into coherent groups. To address this dual clustering requirement, we introduce Spherical Double K-Means (SDKM), a novel methodology that simultaneously clusters documents and terms. This approach offers several advantages: first, by integrating the clustering of documents and terms, SDKM provides deeper insights into the relationships between content and vocabulary, enabling more effective topic identification and keyword extraction. Additionally, the two-level clustering assists in understanding both overarching themes and specific terminologies within document clusters, enhancing interpretability. SDKM effectively handles the high dimensionality and sparsity inherent in text data by utilizing cosine similarity, leading to improved computational efficiency. Moreover, the method captures dynamic changes in thematic content over time, making it well-suited for applications in rapidly evolving fields. Ultimately, SDKM presents a comprehensive framework for advancing text mining efforts, facilitating the uncovering of nuanced patterns and structures that are critical for robust data analysis. We apply SDKM to the corpus of US presidential inaugural addresses, spanning from George Washington in 1789 to Joe Biden in 2021. Our analysis reveals distinct clusters of words and documents that correspond to significant historical themes and periods, showcasing the method's ability to facilitate a deeper understanding of the data. Our findings demonstrate the efficacy of SDKM in uncovering underlying patterns in textual data.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑