arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2409.03203cs.CLcs.AI
收藏

Diffusion LM 在低资源情感分类中用于数据增强的有效部署

An Effective Deployment of Diffusion LM for Data Augmentation in Low-Resource Sentiment Classification

Guangdong University of Foreign Studies(广东外语外贸大学);Guangzhou Key Laboratory of Multilingual Intelligent Processing(广州多语种智能处理重点实验室)

Zhuowei Chen, Lianxi Wang, Yuben Wu, Xinfeng Liao, Yujia Tian, Junyang Zhong

AI总结:

针对低资源情感分类数据增强难以平衡多样性与一致性的问题,提出DiffusionCLS框架,利用扩散语言模型重构强标签相关token生成伪样本并结合抗噪训练,有效提升了模型在多种低资源场景下的性能。

摘要:

情感分类(SC)经常面临低资源挑战,如领域特定上下文、标签分布不平衡和少样本场景。此外,扩散语言模型(LM)在文本数据增强(DA)中的潜力尚未被探索,文本 DA 方法也难以平衡新样本的多样性与一致性。大多数 DA 方法要么进行逻辑修改,要么使用语言模型重写原始序列中不太重要的 token。在 SC 的语境中,强情感 token 对整个序列的情感起着关键作用。因此,与重写不太重要的上下文不同,我们提出 DiffusionCLS,利用扩散 LM 捕获领域内知识,并通过重构强标签相关 token 来生成伪样本。该方法确保了一致性与多样性之间的平衡,避免了噪声的引入并增强了数据集的关键特征。DiffusionCLS 还包含一个抗噪训练目标以帮助模型泛化。实验证明了我们的方法在包括领域特定和领域通用问题在内的各种低资源场景中的有效性。消融研究证实了我们框架模块的有效性,可视化研究突出了最佳部署条件,进一步强化了我们的结论。

英文摘要:

Sentiment classification (SC) often suffers from low-resource challenges such as domain-specific contexts, imbalanced label distributions, and few-shot scenarios. The potential of the diffusion language model (LM) for textual data augmentation (DA) remains unexplored, moreover, textual DA methods struggle to balance the diversity and consistency of new samples. Most DA methods either perform logical modifications or rephrase less important tokens in the original sequence with the language model. In the context of SC, strong emotional tokens could act critically on the sentiment of the whole sequence. Therefore, contrary to rephrasing less important context, we propose DiffusionCLS to leverage a diffusion LM to capture in-domain knowledge and generate pseudo samples by reconstructing strong label-related tokens. This approach ensures a balance between consistency and diversity, avoiding the introduction of noise and augmenting crucial features of datasets. DiffusionCLS also comprises a Noise-Resistant Training objective to help the model generalize. Experiments demonstrate the effectiveness of our method in various low-resource scenarios including domain-specific and domain-general problems. Ablation studies confirm the effectiveness of our framework's modules, and visualization studies highlight optimal deployment conditions, reinforcing our conclusions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑