PerSo: Investigating Facets of LLMs' Social Intelligence in Real-World Persuasion Scenarios
School of Electrical and Computer Engineering, College of Engineering, University of Tehran(德黑兰大学工程学院电气与计算机工程学院);Cardiff University(卡迪夫大学);Institute for Research in Fundamental Sciences (IPM)(基础科学研究所);Tehran Institute for Advanced Studies, Khatam University(哈塔姆大学德黑兰高级研究院)
Ali Zamani, Soroush Esfahanian, Erfan Shahabi, Mohammad Taher Pilehvar, Azadeh Shakery, Yadollah Yaghoobzadeh
备注:
25 pages, 17 figures, 10 tables. Ali Zamani and Soroush Esfahanian contributed equally
Understanding the social intelligence of large language models (LLMs) is increasingly important for applications involving human-AI interaction and multi-agent environments. Social intelligence comprises two aspects: cognitive intelligence, the ability to infer others' mental states, and behavioral intelligence, the ability to select appropriate social actions based on these inferences to achieve social goals. Persuasion provides a natural testbed in which both aspects are simultaneously at play, yet prior benchmarks typically evaluate them in isolation using synthetic scenarios. We introduce PerSo, a benchmark that evaluates both aspects on real-world persuasion dialogues from Reddit's ChangeMyView community, using an LLM-in-the-loop pipeline to construct the evaluation tasks, validated through human annotation. Our evaluation of diverse LLMs shows that performance generally scales with model size but varies across dimensions. The strongest LLM slightly surpasses the best observed human performance on the behavioral task, while the best human participant remains ahead on the cognitive task.
The Dark Patterns of Personalized Persuasion in Large Language Models: Exposing Persuasive Linguistic Features for Big Five Personality Traits in LLMs Responses
Warsaw University of Technology(华沙理工大学);University of Warsaw(华沙大学)
BIG5-TPoT: Predicting BIG Five Personality Traits, Facets, and Items Through Targeted Preselection of Texts
The University of Maryland Applied Research Laboratory for Intelligence and Security (ARLIS)(马里兰大学应用研究实验室(情报与安全实验室));The University of Maryland (UMD)(马里兰大学)
The Levers of Political Persuasion with Conversational AI
UK AI Security Institute(英国人工智能安全研究所);University of Oxford(牛津大学);The London School of Economics and Political Science(伦敦政治经济学院);Stanford University(斯坦福大学);Massachusetts Institute of Technology(麻省理工学院)
MindShift: Leveraging Large Language Models for Mental-States-Based Problematic Smartphone Use Intervention
Tsinghua University(清华大学);University of Washington(华盛顿大学);Beijing University of Posts and Telecommunications(北京邮电大学);Massachusetts Institute of Technology(麻省理工学院)
Automatic Item Generation for Personality Situational Judgment Tests with Large Language Models
Beijing Normal University(北京师范大学);Central China Normal University(华中师范大学);Key Laboratory of Adolescent Cyberpsychology and Behavior, Ministry of Education(教育部青少年网络心理与行为重点实验室);Beijing Key Laboratory of Applied Experimental Psychology(北京市应用实验心理学重点实验室);Research Center for Capacity Building in Educational Assessment and Evaluation (Beijing Higher Education Innovation Center for Philosophy and Social Sciences)(教育评价与评估能力建设研究中心(北京高等教育哲学社会科学创新中心))
Personalization of Large Language Models: A Survey
Dartmouth College(达特茅斯学院);Adobe Research(Adobe研究);Stanford University(斯坦福大学);University of Massachusetts Amherst(马萨诸塞大学阿默斯特分校);Pattern Data;Vanderbilt University(范德比尔特大学);Dolby Research(Dolby研究);University of California San Diego(加州大学圣地亚哥分校);Cisco Research(思科研究);University of Oregon(俄勒冈大学)
Transformative Influence of LLM and AI Tools in Student Social Media Engagement: Analyzing Personalization, Communication Efficiency, and Collaborative Learning