arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2403.14006cs.CLcs.AI
收藏

关于ChatGPT在情感计算中的提示敏感性研究

On Prompt Sensitivity of ChatGPT in Affective Computing

University of Augsburg(奥格斯堡大学);SYNCPILOT GmbH(SYNCPILOT有限公司);Imperial College London(伦敦帝国学院)

Mostafa M. Amin, Björn W. Schuller

AI总结:

本研究提出一种评估方法,通过分析温度和top-$p$等生成参数及不同提示结构对ChatGPT在情感分析、毒性检测和讽刺检测任务中的性能敏感性,探讨其在情感计算领域的提示有效性。

摘要:

近期研究表明,像ChatGPT这样的基础模型在包括情感计算在内的多个领域展现出了新兴能力。然而,获取这些新兴能力需要通过提示工程来实现。尽管已存在一些提示技术,该领域仍在快速发展,许多提示思路仍有待研究。在本研究中,我们提出了一种方法,用于评估和调查基础模型在不同提示或生成参数下的性能敏感性。我们在情感计算范围内,针对情感分析、毒性检测和讽刺检测这三个主要问题,对ChatGPT进行了评估。首先,我们对自回归文本生成中的关键参数进行了敏感性分析,特别是温度参数$T$和Nucleus采样中的top-$p$参数,这些参数决定了模型在生成过程中应保持保守还是创造性。此外,我们探讨了几种提示思路的有效性,研究了给予不同激励或结构如何影响性能。我们的评估考虑了情感计算任务上的性能指标,以及模型遵循既定指令的有效性,从而生成易于解析的响应,以便在下游应用中顺利使用。

英文摘要:

Recent studies have demonstrated the emerging capabilities of foundation models like ChatGPT in several fields, including affective computing. However, accessing these emerging capabilities is facilitated through prompt engineering. Despite the existence of some prompting techniques, the field is still rapidly evolving and many prompting ideas still require investigation. In this work, we introduce a method to evaluate and investigate the sensitivity of the performance of foundation models based on different prompts or generation parameters. We perform our evaluation on ChatGPT within the scope of affective computing on three major problems, namely sentiment analysis, toxicity detection, and sarcasm detection. First, we carry out a sensitivity analysis on pivotal parameters in auto-regressive text generation, specifically the temperature parameter $T$ and the top-$p$ parameter in Nucleus sampling, dictating how conservative or creative the model should be during generation. Furthermore, we explore the efficacy of several prompting ideas, where we explore how giving different incentives or structures affect the performance. Our evaluation takes into consideration performance measures on the affective computing tasks, and the effectiveness of the model to follow the stated instructions, hence generating easy-to-parse responses to be smoothly used in downstream applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑