Conversational systems are now capable of producing impressive and generally relevant responses. However, we have no visibility nor control of the socio-emotional strategies behind state-of-the-art Large Language Models (LLMs), which poses a problem in terms of their transparency and thus their trustworthiness for critical applications. Another issue is that current automated metrics are not able to properly evaluate the quality of generated responses beyond the dataset's ground truth. In this paper, we propose a neural architecture that includes an intermediate step in planning socio-emotional strategies before response generation. We compare the performance of open-source baseline LLMs to the outputs of these same models augmented with our planning module. We also contrast the outputs obtained from automated metrics and evaluation results provided by human annotators. We describe a novel evaluation protocol that includes a coarse-grained consistency evaluation, as well as a finer-grained annotation of the responses on various social and emotional criteria. Our study shows that predicting a sequence of expected strategy labels and using this sequence to generate a response yields better results than a direct end-to-end generation scheme. It also highlights the divergences and the limits of current evaluation metrics for generated content. The code for the annotation platform and the annotated data are made publicly available for the evaluation of future models.
Topics as Proxies for Sociodemographics: How Conversational Context Affects LLM Answers
Institute for Logic, Language and Computation, University of Amsterdam（逻辑、语言与计算研究所，阿姆斯特丹大学）；Khoury College of Computer Sciences, Northeastern University（计算机科学学院，东北大学）；Center for Language and Cognition, University of Groningen（语言与认知中心，格罗宁根大学）
A$^2$-LLM: An End-to-end Conversational Audio Avatar Large Language Model
State Key Laboratory of Information Photonics（信息光子学国家重点实验室）；Optical Communications, Beijing University of Posts（邮电大学光学通信学院）；Zhongguancun Institute of Artificial Intelligence, Beijing, China（中关村人工智能研究院）；School of Finance and Statistics, East China Normal University, Shanghai, China（东华大学金融与统计学院）
The Effect of Emotional Context on Large Language Models' Endorsement of Premature Decisions: Comparing Emotional Vulnerability Across Six Commercial Models
Yonsei University（延世大学）；Fudan University（复旦大学）；St. Johnsbury Academy Jeju（济州圣约翰斯伯里学院）
BenSyc: Benchmarking Conversational Sycophancy and Human Alignment in LLMs for Bengali Contexts
University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Virginia Tech（弗吉尼亚理工大学）；Bangladesh University of Engineering and Technology（孟加拉工程与技术大学）；BRAC University（BRAC大学）；University of Toronto（多伦多大学）
EiCAP: Beyond Fluency, Probing and Improving Emotional Intelligence in LLMs via Psychologically Grounded Multi-Turn Dialogue
Qatar Computing Research Institute (QCRI), Hamad Bin Khalifa University（卡塔尔计算研究所（QCRI），哈马德·本·卡伊夫大学）；University of Illinois Urbana-Champaign (UIUC)（伊利诺伊大学厄巴纳-香槟分校）
CAPE: A Chinese Dataset for Appraisal-based Emotional Generation using Large Language Models
International Digital Economy Academy (IDEA)（国际数字经济学院（IDEA））；University of Michigan（密歇根大学）；Hong Kong University of Science and Technology（香港科技大学）；University of California, Los Angeles（加州大学洛杉矶分校）
Foviatech GmbH（Foviatech有限公司）；University of Hamburg（汉堡大学）
Many socio-linguistic cues are used in conversational analysis, such as emotion, sentiment, and dialogue acts. One of the fundamental cues is politeness, which linguistically possesses properties such as social manners useful in conversational analysis. This article presents findings of polite emotional dialogue act associations, where we can correlate the relationships between the socio-linguistic cues. We confirm our hypothesis that the utterances with the emotion classes Anger and Disgust are more likely to be impolite. At the same time, Happiness and Sadness are more likely to be polite. A…
RESCUE-BENCH: Towards Relation-Aware Multi-Party Emotional Support Conversation Systems
Hong Kong Polytechnic University（香港理工大学）；Nanjing University of Science and Technology（南京理工大学）；XU Exponential University of Applied Sciences（XU应用科学大学）；Vrije Universiteit Amsterdam（阿姆斯特丹自由大学）
University of Hamburg（汉堡大学）；Foviatech GmbH（Foviatech 有限公司）
Many socio-linguistic cues are used in the conversational analysis, such as emotion, sentiment, and dialogue acts. One of the fundamental social cues is politeness, which linguistically possesses properties useful in conversational analysis. This short article presents some of the brief findings of polite emotional dialogue acts, where we can correlate the relational bonds between these socio-linguistics cues. We found that the utterances with emotion classes Anger and Disgust are more likely to be impolite while Happiness and Sadness to be polite. Similar phenomenon occurs with dialogue acts,…
TRACE: Temporal Relationship-Aware Conversational Entrainment Detection in Dyadic Speech
Department of Computer Science, Johns Hopkins University（约翰霍普金斯大学计算机科学系）；Department of Electrical and Computer Engineering, Johns Hopkins University（约翰霍普金斯大学电气与计算机工程系）；Amazon Research（亚马逊研究院）；Amazon（亚马逊）
South China University of Technology（华南理工大学）；UBTECH Robotics Corp（优必选科技股份有限公司）；Tsinghua University（清华大学）
Human language expression is based on the subjective construal of the situation instead of the objective truth conditions, which means that speakers' personalities and emotions after cognitive processing have an important influence on conversation. However, most existing datasets for conversational AI ignore human personalities and emotions, or only consider part of them. It's difficult for dialogue systems to understand speakers' personalities and emotions although large-scale pre-training language models have been widely used. In order to consider both personalities and emotions in the proce…
An Offline Mobile Conversational Agent for Mental Health Support: Learning from Emotional Dialogues and Psychological Texts with Student-Centered Evaluation
Department of Electrical Engineering and Computer Science, Indian Institute of Science Education and Research Bhopal（电子工程与计算机科学系，比哈尔印度科学教育与研究中心）
