arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2303.16764cs.CL
收藏

基于分布估计的Few-Shot文本分类

Boosting Few-Shot Text Classification via Distribution Estimation

Dalian University of Technology(大连理工大学);Peking University(北京大学);The Pennsylvania State University(宾夕法尼亚州立大学);The Hong Kong Polytechnic University(香港理工大学);Zhejiang Lab(之江实验室)

Han Liu, Feng Zhang, Xiaotong Zhang, Siyang Zhao, Fenglong Ma, Xiao-Ming Wu, Hongyang Chen, Hong Yu, Xianchao Zhang

摘要:

在计算机视觉领域中,分布估计是处理Few-Shot图像分类的最有效方法之一,因为低层模式和底层表示可以在不同的任务之间容易地传递。然而,直接将这种方法应用于Few-Shot文本分类是具有挑战性的,因为利用具有足够样本的已知类的统计来校准新类的分布可能会由于文本域中严重的类别差异而导致负面影响。为了缓解这个问题,我们提出了两个简单而有效的策略来估计新的类的分布,利用未标记的查询样本,从而避免潜在的负迁移问题。具体来说,我们首先假设一个类或样本服从高斯分布,并使用原始支持集和最近的几个查询样本来估计相应的均值和协方差。然后,我们通过从估计的分布中采样来增加标记样本,这可以为训练分类模型提供足够的监督。在8个Few-Shot文本分类数据集上的实验表明,该方法的分类性能明显优于现有的基线方法。

英文摘要:

Distribution estimation has been demonstrated as one of the most effective approaches in dealing with few-shot image classification, as the low-level patterns and underlying representations can be easily transferred across different tasks in computer vision domain. However, directly applying this approach to few-shot text classification is challenging, since leveraging the statistics of known classes with sufficient samples to calibrate the distributions of novel classes may cause negative effects due to serious category difference in text domain. To alleviate this issue, we propose two simple yet effective strategies to estimate the distributions of the novel classes by utilizing unlabeled query samples, thus avoiding the potential negative transfer issue. Specifically, we first assume a class or sample follows the Gaussian distribution, and use the original support set and the nearest few query samples to estimate the corresponding mean and covariance. Then, we augment the labeled samples by sampling from the estimated distribution, which can provide sufficient supervision for training the classification model. Extensive experiments on eight few-shot text classification datasets show that the proposed method outperforms state-of-the-art baselines significantly.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑