arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1507.01529cs.CL
收藏

大数据集的对应因子分析:3000万词的案例研究;以及使用Apache Solr和R中的对应分析的对比分析

Correspondence Factor Analysis of Big Data Sets: A Case Study of 30 Million Words; and Contrasting Analytics using Apache Solr and Correspondence Analysis in R

Fionn Murtagh

摘要:

我们考虑了大量的文本数据集。这些是烹饪食谱。项分布和其他分布特性的数据进行了研究。我们的目的是看看各种分析方法,允许挖掘的信息,高和低的细节尺度。度量空间嵌入是我们对这些数据的语义属性感兴趣的基础。我们考虑将所有数据投影到数据的汇总版本的分析中。我们将其与将数据的聚合版本投影到所有数据的分析中进行对比。类似地,对于术语集,我们查看对所选术语的分析。我们还将研究术语的固有关联,例如单数和复数之间的关联。除了使用R中的对应分析,对于潜在语义空间映射,我们还使用Apache Solr。描述了Solr服务器的建立和查询。另一个新颖之处在于,Solr基于所有数据的主因子平面映射来支持查询。它使用基于因子投影的限界框查询。

英文摘要:

We consider a large number of text data sets. These are cooking recipes. Term distribution and other distributional properties of the data are investigated. Our aim is to look at various analytical approaches which allow for mining of information on both high and low detail scales. Metric space embedding is fundamental to our interest in the semantic properties of this data. We consider the projection of all data into analyses of aggregated versions of the data. We contrast that with projection of aggregated versions of the data into analyses of all the data. Analogously for the term set, we look at analysis of selected terms. We also look at inherent term associations such as between singular and plural. In addition to our use of Correspondence Analysis in R, for latent semantic space mapping, we also use Apache Solr. Setting up the Solr server and carrying out querying is described. A further novelty is that querying is supported in Solr based on the principal factor plane mapping of all the data. This uses a bounding box query, based on factor projections.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑