arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2307.03952cs.CL

ChatGPT 是一个好的性格识别器吗?一项初步研究

Is ChatGPT a Good Personality Recognizer? A Preliminary Study

Institute of AI Education, East China Normal University(华东师范大学人工智能教育研究院);School of Computer Science and Technology, East China Normal University(华东师范大学计算机科学与技术学院);School of Psychology and Cognitive Science, East China Normal University(华东师范大学心理与认知科学学院);Shanghai Changning Mental Health Center(上海市长宁区精神卫生中心)

Yu Ji, Wen Wu, Hong Zheng, Yi Hu, Xi Chen, Liang He

AI总结:

本研究初步评估了 ChatGPT 在文本性格识别任务上的能力,发现零样本思维链提示效果显著,并设计面向级别的提示策略缩小了与最先进模型的差距,同时揭示了其对性别、年龄的偏见及对下游任务的提升作用。

摘要:

近年来,性格被视为一种有价值的个人因素,被纳入情感分析和产品推荐等众多任务中。这使得基于文本的性格识别任务受到广泛关注,该任务旨在根据给定文本识别个体的性格。考虑到 ChatGPT 最近在各种自然语言处理任务中展现出卓越的能力,我们针对基于文本的性格识别任务对 ChatGPT 进行了初步评估,以生成有效的性格数据。具体而言,我们采用了多种提示策略来探索 ChatGPT 从给定文本中识别性格的能力,尤其是我们设计的面向级别的提示策略,用于引导 ChatGPT 在指定级别上分析给定文本。在两个具有代表性的真实世界数据集上的实验结果表明,采用零样本思维链提示的 ChatGPT 展现出令人印象深刻的性格识别能力,并能够通过基于文本的逻辑推理提供自然语言解释。此外,通过采用面向级别的提示策略来优化零样本思维链提示,ChatGPT 与相应最先进模型之间的性能差距进一步缩小。然而,我们观察到 ChatGPT 对某些敏感的人口统计属性(如性别和年龄)表现出不公平性。此外,我们发现激发 ChatGPT 的性格识别能力有助于提升其在性格相关下游任务(如情感分类和压力预测)上的表现。

英文摘要:

In recent years, personality has been regarded as a valuable personal factor being incorporated into numerous tasks such as sentiment analysis and product recommendation. This has led to widespread attention to text-based personality recognition task, which aims to identify an individual's personality based on given text. Considering that ChatGPT has recently exhibited remarkable abilities on various natural language processing tasks, we provide a preliminary evaluation of ChatGPT on text-based personality recognition task for generating effective personality data. Concretely, we employ a variety of prompting strategies to explore ChatGPT's ability in recognizing personality from given text, especially the level-oriented prompting strategy we designed for guiding ChatGPT in analyzing given text at a specified level. The experimental results on two representative real-world datasets reveal that ChatGPT with zero-shot chain-of-thought prompting exhibits impressive personality recognition ability and is capable to provide natural language explanations through text-based logical reasoning. Furthermore, by employing the level-oriented prompting strategy to optimize zero-shot chain-of-thought prompting, the performance gap between ChatGPT and corresponding state-of-the-art model has been narrowed even more. However, we observe that ChatGPT shows unfairness towards certain sensitive demographic attributes such as gender and age. Additionally, we discover that eliciting the personality recognition ability of ChatGPT helps improve its performance on personality-related downstream tasks such as sentiment classification and stress prediction.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑