ChatGPT在情感计算任务上的广泛评估
A Wide Evaluation of ChatGPT on Affective Computing Tasks
AI总结：
本研究评估了GPT-3.5、GPT-4在13类情感计算任务上的能力，提出将回归类任务转化为成对排序分类的评估框架，发现其在多数情感相关任务上表现优异，但在隐含信号类任务上存在不足。
摘要：
随着基础模型的兴起，一种新的人工智能范式已经出现：只需使用带提示工程的通用基础模型来解决问题，而非为每个问题单独训练机器学习模型。这类模型已被证明具备解决其最初未被训练过的问题的涌现能力，目前关于这类模型有效性的研究仍十分有限。\n本研究广泛探究了ChatGPT模型（即GPT-4与GPT-3.5）在13项情感计算问题上的能力，这些问题包括：方面提取、方面极性分类、观点提取、情感分析、情感强度排序、情绪强度排序、自杀倾向检测、毒性检测、幸福感评估、投入度测量、人格评估、讽刺检测和主观性检测。\n我们提出了一个框架，用于评估ChatGPT模型在基于回归的问题（如强度排序问题）上的表现，方法是将这类问题建模为成对排序分类任务。我们将ChatGPT与更传统的自然语言处理方法（如端到端循环神经网络和Transformer模型）进行了对比。\n结果证明了ChatGPT模型在广泛情感计算问题上的涌现能力：GPT-3.5，尤其是GPT-4，在诸多问题上表现出色，特别是与情感、情绪或毒性相关的问题。而在涉及隐含信号的问题（如投入度测量和主观性检测）上，ChatGPT模型的表现有所欠缺。
英文摘要：
With the rise of foundation models, a new artificial intelligence paradigm has emerged, by simply using general purpose foundation models with prompting to solve problems instead of training a separate machine learning model for each problem. Such models have been shown to have emergent properties of solving problems that they were not initially trained on. The studies for the effectiveness of such models are still quite limited. In this work, we widely study the capabilities of the ChatGPT models, namely GPT-4 and GPT-3.5, on 13 affective computing problems, namely aspect extraction, aspect polarity classification, opinion extraction, sentiment analysis, sentiment intensity ranking, emotions intensity ranking, suicide tendency detection, toxicity detection, well-being assessment, engagement measurement, personality assessment, sarcasm detection, and subjectivity detection. We introduce a framework to evaluate the ChatGPT models on regression-based problems, such as intensity ranking problems, by modelling them as pairwise ranking classification. We compare ChatGPT against more traditional NLP methods, such as end-to-end recurrent neural networks and transformers. The results demonstrate the emergent abilities of the ChatGPT models on a wide range of affective computing problems, where GPT-3.5 and especially GPT-4 have shown strong performance on many problems, particularly the ones related to sentiment, emotions, or toxicity. The ChatGPT models fell short for problems with implicit signals, such as engagement measurement and subjectivity detection.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
