Emotion Cause Extraction (ECE) aims to identify emotion causes from a document after annotating the emotion keywords. Some baselines have been proposed to address this problem, such as rule-based, commonsense based and machine learning methods. We show, however, that a simple random selection approach toward ECE that does not require observing the text achieves similar performance compared to the baselines. We utilized only position information relative to the emotion cause to accomplish this goal. Since position information alone without observing the text resulted in higher F-measure, we therefore uncovered a bias in the ECE single genre Sina-news benchmark. Further analysis showed that an imbalance of emotional cause location exists in the benchmark, with a majority of cause clauses immediately preceding the central emotion clause. We examine the bias from a linguistic perspective, and show that high accuracy rate of current state-of-art deep learning models that utilize location information is only evident in datasets that contain such position biases. The accuracy drastically reduced when a dataset with balanced location distribution is introduced. We therefore conclude that it is the innate bias in this benchmark that caused high accuracy rate of these deep learning models in ECE. We hope that the case study in this paper presents both a cautionary lesson, as well as a template for further studies, in interpreting the superior fit of deep learning models without checking for bias.
How people talk about each other: Modeling Generalized Intergroup Bias and Emotion
Department of Linguistics, The University of Texas at Austin(德克萨斯大学奥斯汀分校语言学系);Department of Computer Sciences, University of Pittsburgh(匹兹堡大学计算机科学系);Department of Political Science, Rutgers University(罗格斯大学政治学系)
Us vs. Them: A Dataset of Populist Attitudes, News Bias and Emotions
Institute for Logic, Language and Computation, University of Amsterdam(阿姆斯特丹大学逻辑、语言与计算研究所);Babelscape Srl, Sapienza University of Rome(罗马第一大学Babelscape有限公司);Department of Psychology, University of Amsterdam(阿姆斯特丹大学心理学系)
政治语篇任务的计算建模已经成为自然语言处理中越来越重要的研究领域。近年来,民粹主义言论在整个政治领域有所上升;然而,由于其复杂性,对其的计算方法一直很稀缺。在本文中,我们提出了新的$\textit{Us vs. Them}$数据集,包括6861条Reddit评论,注释了民粹主义的态度和这种现象的第一个大规模计算模型。我们调查了民粹主义心态和社会群体之间的关系,以及一系列通常与这些相关的情绪。我们设定了一个基线的两个任务相关的民粹主义的态度,并提出了一套多任务的学习模型,利用和展示的重要性,情绪和群体识别作为辅助任务。
Multi-Granularity Semantic Aware Graph Model for Reducing Position Bias in Emotion-Cause Pair Extraction
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院)
Can Induced Emotion Bias LLM Behaviors in Sequential Decision Making?
MBZUAI(穆罕默德·本·扎耶德人工智能大学);Texas A&M University(德州农工大学);National University of Singapore(新加坡国立大学);UCLA(加州大学洛杉矶分校);University of Texas at Austin(德克萨斯大学奥斯汀分校);Mass General Hospital(麻省总医院);Harvard Medical School(哈佛医学院)
High Risk of Political Bias in Black Box Emotion Inference Models
Institute of Psychology, Polish Academy of Sciences(波兰科学院心理学研究所);Stowarzyszenie Otwartej Nauki(开放科学协会);Institute of Philosophy and Sociology, Polish Academy of Sciences(波兰科学院哲学与社会学研究所)
Position Bias Mitigates Position Bias:Mitigate Position Bias Through Inter-Position Knowledge Distillation
MAIS, Institute of Automation, Chinese Academy of Sciences(信息与自动化研究所,中国科学院);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);AMAP, Alibaba Group(AMAP,阿里巴巴集团)