面向短文本多标签情感分析的小型Transformer语言模型优化
Optimizing Small Transformer-Based Language Models for Multi-Label Sentiment Analysis in Short Texts
AI总结：
针对短文本多标签情感分类的挑战，研究评估了领域预训练、生成式数据增强、分类头架构三类优化策略对小型Transformer模型的影响，明确了各策略的实际效果，为资源受限场景下的短文本情感分类优化提供指导。
摘要：
短文本数据集上的情感分类面临类别不平衡、训练样本有限以及情感标签固有的主观性等重大挑战，而短文本的有限上下文会进一步加剧这些问题。这些因素导致歧义难以消解，还会加剧数据稀疏性，阻碍模型的有效学习。本文评估了基于Transformer的小型模型（即参数量少于10亿的BERT和RoBERTa）在多标签情感分类任务中的效果，重点关注短文本场景。具体而言，我们评估了影响模型性能的三个关键因素：（1）持续的领域特定预训练；（2）使用自动生成样本的数据增强方法，即生成式数据增强；（3）分类头的架构变体。实验结果表明，数据增强能够提升分类性能，而在增强后的数据集上进行持续预训练不仅无法提高准确率，反而会引入噪声。此外，我们证实对分类头的修改仅能带来边际收益。这些发现为在资源受限场景下优化基于BERT的模型，以及完善短文本数据集的情感分类策略提供了实用指导。
英文摘要：
Sentiment classification in short text datasets faces significant challenges such as class imbalance, limited training samples, and the inherent subjectivity of sentiment labels -- issues that are further intensified by the limited context in short texts. These factors make it difficult to resolve ambiguity and exacerbate data sparsity, hindering effective learning. In this paper, we evaluate the effectiveness of small Transformer-based models (i.e., BERT and RoBERTa, with fewer than 1 billion parameters) for multi-label sentiment classification, with a particular focus on short-text settings. Specifically, we evaluated three key factors influencing model performance: (1) continued domain-specific pre-training, (2) data augmentation using automatically generated examples, specifically generative data augmentation, and (3) architectural variations of the classification head. Our experiment results show that data augmentation improves classification performance, while continued pre-training on augmented datasets can introduce noise rather than boost accuracy. Furthermore, we confirm that modifications to the classification head yield only marginal benefits. These findings provide practical guidance for optimizing BERT-based models in resource-constrained settings and refining strategies for sentiment classification in short-text datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
