When humans judge the affective content of texts, they also implicitly assess the correctness of such judgment, that is, their confidence. We hypothesize that people's (in)confidence that they performed well in an annotation task leads to (dis)agreements among each other. If this is true, confidence may serve as a diagnostic tool for systematic differences in annotations. To probe our assumption, we conduct a study on a subset of the Corpus of Contemporary American English, in which we ask raters to distinguish neutral sentences from emotion-bearing ones, while scoring the confidence of their answers. Confidence turns out to approximate inter-annotator disagreements. Further, we find that confidence is correlated to emotion intensity: perceiving stronger affect in text prompts annotators to more certain classification performances. This insight is relevant for modelling studies of intensity, as it opens the question wether automatic regressors or classifiers actually predict intensity, or rather human's self-perceived confidence.
"You are an expert annotator": Automatic Best-Worst-Scaling Annotations for Emotion Intensity Modeling
Merck Data & AI Organization, Merck Group（默克集团默克数据与人工智能部门）；Institut für Maschinelle Sprachverarbeitung, University of Stuttgart（斯图加特大学机器语言处理研究所）；Fundamentals of Natural Language Processing, University of Bamberg（班贝格大学自然语言处理基础研究所）；Merck IT Centre, Merck Group（默克集团默克IT中心）
Extracting Consumer Insight from Text: A Large Language Model Approach to Emotion and Evaluation Measurement
Monash University（莫纳什大学）；University of New South Wales（新南威尔士大学）；Flinders University（弗林德斯大学）；Babson College（巴布森学院）；University of Bath（巴斯大学）；Tecnológico de Monterrey（蒙特雷理工学院）
How to predict creativity ratings from written narratives: A comparison of co-occurrence and textual forma mentis networks
Center for Mind/Brain Sciences (CIMeC), University of Trento（心灵/大脑科学中心（CIMeC），特伦托大学）；CogNosco Lab, Department of Psychology and Cognitive Science, University of Trento（CogNosco实验室，心理学与认知科学系，特伦托大学）
Emotion Recognition for Vietnamese Social Media Text
University of Information Technology, VNU-HCM（胡志明市信息技术大学（VNU-HCM））；Vietnam National University, Ho Chi Minh City（越南胡志明市国家大学）；University of Social Sciences and Humanities, VNU-HCM（胡志明市社会科学与人文大学（VNU-HCM））；Multimedia Communications Laboratory, University of Information Technology, VNU-HCM（胡志明市信息技术大学多媒体通信实验室（VNU-HCM））
AI shares emotion with humans across languages and cultures
University of Science and Technology of China（中国科学技术大学）；Peking University（北京大学）；Tsinghua University（清华大学）；Beijing Normal University（北京师范大学）；The Hong Kong Polytechnic University（香港理工大学）；The PolyU-Hangzhou Technology and Innovation Research Institute（香港理工大学杭州科技创新研究院）；Changping Laboratory（昌平实验室）
Bridging the behavior-neural gap: A multimodal AI reveals the brain's geometry of emotion more accurately than human self-reports
State Key Laboratory of Brain Cognition and Brain-inspired Intelligence Technology, Institute of Automation, Chinese Academy of Sciences（脑认知与脑启发智能技术重点实验室，自动化研究所，中国科学院）；School of Artificial Intelligence, University of Chinese Academy of Sciences（人工智能学院，中国科学院大学）；School of Future Technology, University of Chinese Academy of Sciences（未来技术学院，中国科学院大学）；State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University（认知神经科学与学习国家重点实验室，北京师范大学）
