有监督项加权中过权与欠权的平衡
Balancing Between Over-Weighting and Under-Weighting in Supervised Term Weighting
摘要:
有监督的词加权可以提高文本分类的性能。一种被证明是有效的方法是给那些在类别中分布更不平衡的术语更多的权重。本文表明,监督的术语加权不应该只分配大的权重不平衡的条款,但也应该控制之间的权衡过度加权和不足的权重。本文提出了一个新的概念--超权,它是由于对奇异项处理不当和项权比值过大而引起的。为了防止权重过大,我们提出了三种正则化技术:加一平滑,次线性缩放和偏置项。加一平滑用于处理奇异项。次线性缩放和偏置项缩小了项权重之间的比率。然而,如果次线性函数按比例缩小项权重太多,或者偏置项太大,则会发生欠权重并损害性能。因此,在权重过高和权重过低之间取得平衡至关重要。受此启发,我们还提出了一种新的监督项加权方案,正则化熵(re)。本文采用熵来度量术语分布,并引入偏差项来控制权重的过小。在主题和情感分类数据集上的实验结果表明,次线性尺度和偏置项对监督项加权的性能有很大影响,与现有方案相比,我们的方案效果最好.
英文摘要:
Supervised term weighting could improve the performance of text categorization. A way proven to be effective is to give more weight to terms with more imbalanced distributions across categories. This paper shows that supervised term weighting should not just assign large weights to imbalanced terms, but should also control the trade-off between over-weighting and under-weighting. Over-weighting, a new concept proposed in this paper, is caused by the improper handling of singular terms and too large ratios between term weights. To prevent over-weighting, we present three regularization techniques: add-one smoothing, sublinear scaling and bias term. Add-one smoothing is used to handle singular terms. Sublinear scaling and bias term shrink the ratios between term weights. However, if sublinear functions scale down term weights too much, or the bias term is too large, under-weighting would occur and harm the performance. It is therefore critical to balance between over-weighting and under-weighting. Inspired by this insight, we also propose a new supervised term weighting scheme, regularized entropy (re). Our re employs entropy to measure term distribution, and introduces the bias term to control over-weighting and under-weighting. Empirical evaluations on topical and sentiment classification datasets indicate that sublinear scaling and bias term greatly influence the performance of supervised term weighting, and our re enjoys the best results in comparison with existing schemes.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。