一种基于规则/BPSO的低维语义基向量集生成方法
A Rule-based/BPSO Approach to Produce Low-dimensional Semantic Basis Vectors Set
摘要:
我们打算生成低维显式分布语义向量。在显式语义向量中,每个维度对应一个词,因此词向量是可解释的。在这项研究中,我们提出了一种新的方法来获得低维显式语义向量。首先,所提出的方法认为三个标准的词相似度,零的数量,和词频作为特征的词在语料库中。然后,我们提取一些规则,获得初始的基础词使用的决策树,这是基于这三个功能。其次,我们提出了一种基于二进制粒子群优化算法的二进制加权方法,获得N_B = 1000个上下文单词。我们还使用了一个词选择方法,提供N_S = 1000上下文单词。第三,基于二进制加权法提取语料库中的金词。然后,我们将提取的黄金单词添加到由单词选择方法选择的上下文单词中作为黄金上下文单词。我们使用ukWaC语料库来构建词向量。我们使用MEN,RG-65和SimLex-999测试集来评估词向量。我们报告的结果相比,基线使用5 k最频繁的词在语料库中作为上下文的话。基线方法使用固定的窗口来计数同现。我们使用1000个选择的上下文单词以及黄金上下文单词来获得单词向量。与基线方法相比,我们的方法将MEN、RG-65和SimLex-999测试集的Spearman相关系数分别提高了4.66%、14.73%和1.08%。
英文摘要:
We intend to generate low-dimensional explicit distributional semantic vectors. In explicit semantic vectors, each dimension corresponds to a word, so word vectors are interpretable. In this research, we propose a new approach to obtain low-dimensional explicit semantic vectors. First, the proposed approach considers the three criteria Word Similarity, Number of Zero, and Word Frequency as features for the words in a corpus. Then, we extract some rules for obtaining the initial basis words using a decision tree that is drawn based on the three features. Second, we propose a binary weighting method based on the Binary Particle Swarm Optimization algorithm that obtains N_B = 1000 context words. We also use a word selection method that provides N_S = 1000 context words. Third, we extract the golden words of the corpus based on the binary weighting method. Then, we add the extracted golden words to the context words that are selected by the word selection method as the golden context words. We use the ukWaC corpus for constructing the word vectors. We use MEN, RG-65, and SimLex-999 test sets to evaluate the word vectors. We report the results compared to a baseline that uses 5k most frequent words in the corpus as context words. The baseline method uses a fixed window to count the co-occurrences. We obtain the word vectors using the 1000 selected context words together with the golden context words. Our approach compared to the Baseline method increases the Spearman correlation coefficient for the MEN, RG-65, and SimLex-999 test sets by 4.66%, 14.73%, and 1.08%, respectively.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。