arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.05627cs.CLcs.IRcs.ITcs.LGcs.SImath.IT
收藏

文本分类研究新方向:从有限数据中最大化情感分类性能

New Directions in Text Classification Research: Maximizing The Performance of Sentiment Classification from Limited Data

UIN Sultan Syarif Kasim Riau(苏丹谢里夫·卡西姆国立伊斯兰大学廖内)

Surya Agustian, Muhammad Irfan Syah, Nurul Fatiara, Rahmad Abdillah

AI总结:

针对有限训练数据下情感分类性能不佳的问题,本文围绕特定政治人物任命议题构建三类情感分类基准数据集,提供外部增强数据,给出基于SVM的基线与优化F1分数参考,为小样本文本分类研究提供方向。

摘要:

利益相关方针对各类议题的情感分析(无论是正面还是负面)的需求是速度与准确性。情感分析任务中的一项新挑战是训练数据有限,这往往会导致机器学习模型表现不佳,在测试数据上的性能较差。本文探讨了基于有限训练数据(300至600个样本)将文本分为正面、负面和中性三类的文本分类问题。本文提供了一个基准数据集,用于围绕Kaesang Pangarep被任命为PSI主席这一议题的训练和测试数据。同时提供了用于聚合和增强的外部数据,包含两个数据集:新冠疫苗接种情感主题数据集和一个开放主题数据集。采用的官方评分指标是F1-score,该指标可平衡正面、负面和中性三个类别之间的精确率和召回率。本文提供了基线分数,作为研究人员参考的未优化分类方法的性能水平;还提供了优化分数,作为任何提出的方法所需达到的目标分数参考。基线评分和优化评分均使用SVM方法,该方法被广泛认为是传统机器学习方法中的当前最优水平。基线方法和优化方法取得的F1-score分别为40.83%和51.28%。

英文摘要:

The stakeholders' needs in sentiment analysis for various issues, whether positive or negative, are speed and accuracy. One new challenge in sentiment analysis tasks is the limited training data, which often leads to suboptimal machine learning models and poor performance on test data. This paper discusses the problem of text classification based on limited training data (300 to 600 samples) into three classes: positive, negative, and neutral. A benchmark dataset is provided for training and testing data on the issue of Kaesang Pangarep's appointment as Chairman of PSI. External data for aggregation and augmentation purposes are provided, consisting of two datasets: the topic of Covid Vaccination sentiment and an open topic. The official score used is the F1-score, which balances precision and recall among the three classes, positive, negative, and neutral. A baseline score is provided as a reference for researchers for unoptimized classification methods. The optimized score is provided as a reference for the target score to be achieved by any proposed method. Both scoring (baseline and optimized) use the SVM method, which is widely reported as the state-of-the-art in conventional machine learning methods. The F1-scores achieved by the baseline and optimized methods are 40.83% and 51.28%, respectively.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑