arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2209.06720cs.CLstat.AP
收藏

关于语言聚类:一种非参数统计方法

On Language Clustering: A Non-parametric Statistical Approach

Indian Statistical Institute(印度统计研究所);VIT Bhopal University(VIT博帕尔大学);Jadavpur University(贾达普大学)

Anagh Chattopadhyay, Soumya Sankar Ghosh, Samir Karmakar

AI总结:

本研究提出一种基于非参数统计与数据深度的语言聚类方法,通过MDS将单词类型结构转换为笛卡尔框架,实现异常值检测与无监督/有监督聚类,为非同质语言分类提供稳健分析。

摘要:

任何旨在规范和量化特定现象的方法,都必须包含用于数据分析的稳健统计方法。考虑到这一点,本研究旨在提出可用于非参数非同质数据框架的统计方法,并探讨其在自然语言处理和语言聚类领域的应用。此外,本文讨论了非参数方法在语言数据挖掘和处理中的多种用途。数据深度概念允许在任何维度上进行由中心向外的点排序,从而产生一种不需要任何分布假设的新的非参数多变量统计分析。层级概念被用于历史语言分类和结构化,旨在使用相同的前提将语言组织并聚类为子语系。在这方面,本研究提出了一种基于非参数方法的语系结构构建新方法,该方法由不同语言中单词的类型结构生成,随后使用MDS转换为笛卡尔框架。这种基于统计深度的架构允许使用基于数据深度的方法进行稳健的异常值检测,这对于理解各种边界语言的分类非常有用,并允许重新评估现有的分类系统。其他基于深度的方法也被应用于无监督和有监督聚类等过程。因此,本文概述了可在非参数框架下应用于非同质语言分类系统的程序。

英文摘要:

Any approach aimed at pasteurizing and quantifying a particular phenomenon must include the use of robust statistical methodologies for data analysis. With this in mind, the purpose of this study is to present statistical approaches that may be employed in nonparametric nonhomogeneous data frameworks, as well as to examine their application in the field of natural language processing and language clustering. Furthermore, this paper discusses the many uses of nonparametric approaches in linguistic data mining and processing. The data depth idea allows for the centre-outward ordering of points in any dimension, resulting in a new nonparametric multivariate statistical analysis that does not require any distributional assumptions. The concept of hierarchy is used in historical language categorisation and structuring, and it aims to organise and cluster languages into subfamilies using the same premise. In this regard, the current study presents a novel approach to language family structuring based on non-parametric approaches produced from a typological structure of words in various languages, which is then converted into a Cartesian framework using MDS. This statistical-depth-based architecture allows for the use of data-depth-based methodologies for robust outlier detection, which is extremely useful in understanding the categorization of diverse borderline languages and allows for the re-evaluation of existing classification systems. Other depth-based approaches are also applied to processes such as unsupervised and supervised clustering. This paper therefore provides an overview of procedures that can be applied to nonhomogeneous language classification systems in a nonparametric framework.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑