arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2309.08631cs.CLcs.AIcs.CYcs.HCcs.LGcs.SI

大型语言模型能够推断社交媒体用户的心理倾向

Large Language Models Can Infer Psychological Dispositions of Social Media Users

Columbia University(哥伦比亚大学)

Heinrich Peters, Sandra Matz

AI总结:

研究测试GPT-3.5和GPT-4在零样本场景下从Facebook状态推断大五人格特质,发现其准确率与监督学习模型相当,但存在年龄和性别偏差,并引发伦理关注。

摘要:

大型语言模型(LLMs)在多种任务中展现出越来越类似人类的能力。本文研究ChatGPT等LLMs能否准确推断社交媒体用户的心理倾向,以及这种能力是否因社会人口群体而异。具体而言,我们测试GPT-3.5和GPT-4能否在零样本学习场景下从用户的Facebook状态更新中推导出大五人格特质。结果显示,LLM推断的特质得分与自我报告得分之间的平均相关性为r = .29(范围 = [.22, .33]),这一准确率与专门训练用于推断人格的监督机器学习模型相似。研究还发现,不同年龄组和性别类别的人格推断准确性存在异质性:在几项特质上,对女性和年轻个体的预测更为准确,这表明潜在的训练数据偏差或在线自我表达差异可能导致潜在偏见。LLM从用户生成文本中推断心理倾向的能力,有望为研究人员和从业者提供低成本、可扩展的心理测量评估。一方面,这种普及化可能促进具有高生态效度的大规模研究,并激发个性化服务的创新。另一方面,它也引发了关于用户隐私和自决权的伦理担忧,凸显了制定严格伦理框架和监管的必要性。

英文摘要:

Large Language Models (LLMs) demonstrate increasingly human-like abilities across a wide variety of tasks. In this paper, we investigate whether LLMs like ChatGPT can accurately infer the psychological dispositions of social media users and whether their ability to do so varies across socio-demographic groups. Specifically, we test whether GPT-3.5 and GPT-4 can derive the Big Five personality traits from users' Facebook status updates in a zero-shot learning scenario. Our results show an average correlation of r = .29 (range = [.22, .33]) between LLM-inferred and self-reported trait scores - a level of accuracy that is similar to that of supervised machine learning models specifically trained to infer personality. Our findings also highlight heterogeneity in the accuracy of personality inferences across different age groups and gender categories: predictions were found to be more accurate for women and younger individuals on several traits, suggesting a potential bias stemming from the underlying training data or differences in online self-expression. The ability of LLMs to infer psychological dispositions from user-generated text has the potential to democratize access to cheap and scalable psychometric assessments for both researchers and practitioners. On the one hand, this democratization might facilitate large-scale research of high ecological validity and spark innovation in personalized services. On the other hand, it also raises ethical concerns regarding user privacy and self-determination, highlighting the need for stringent ethical frameworks and regulation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑