Extensive research has recently shown that recurrent neural language models are able to process a wide range of grammatical phenomena. How these models are able to perform these remarkable feats so well, however, is still an open question. To gain more insight into what information LSTMs base their decisions on, we propose a generalisation of Contextual Decomposition (GCD). In particular, this setup enables us to accurately distil which part of a prediction stems from semantic heuristics, which part truly emanates from syntactic cues and which part arise from the model biases themselves instead. We investigate this technique on tasks pertaining to syntactic agreement and co-reference resolution and discover that the model strongly relies on a default reasoning effect to perform these tasks.
Analysing Differences in Persuasive Language in LLM-Generated Text: Uncovering Stereotypical Gender Patterns
Department of Computer Science, Aarhus University(阿arhus大学计算机科学系);AMD Silo AI;University of Tokyo(东京大学);IT University of Copenhagen(哥本哈根IT大学);Department of Computer Science, University of Copenhagen(哥本哈根大学计算机科学系)
taz2024full: Analysing German Newspapers for Gender Bias and Discrimination across Decades
Faculty for Computer Science and Mathematics, Hochschule München University of Applied Sciences(计算机科学与数学系,慕尼黑应用科学大学);Department of Statistics, LMU Munich(统计学系,慕尼黑大学);Munich Center for Machine Learning (MCML), LMU Munich(慕尼黑机器学习中心(MCML),慕尼黑大学)
Voice, Bias, and Coreference: An Interpretability Study of Gender in Speech Translation
Fondazione Bruno Kessler(布鲁诺·科塞拉基金会);University of Trento(特伦托大学);Laboratoire de Linguistique Formelle, Université Paris Cité, CNRS(巴黎城市大学语言学实验室,法国国家科学研究中心)
Does Context Help Mitigate Gender Bias in Neural Machine Translation?
Vicomtech Foundation(维康科技基金会);Basque Research and Technology Alliance (BRTA)(巴斯克研究与技术联盟(BRTA));University of the Basque Country UPV/EHU(巴斯克大学(UPV/EHU))
Gender Disambiguation in Machine Translation: Diagnostic Evaluation in Decoder-Only Architectures
Research Center for Cognitive Science and Artificial Intelligence(认知科学与人工智能研究中心);Tilburg School of Humanities and Digital Sciences(蒂尔堡人文与数字科学学院);Tilburg University(蒂尔堡大学)
ChatGPT Perpetuates Gender Bias in Machine Translation and Ignores Non-Gendered Pronouns: Findings across Bengali and Five other Low-Resource Languages
Mind the Inclusivity Gap: Multilingual Gender-Neutral Translation Evaluation with mGeNTE
Fondazione Bruno Kessler(布鲁诺·科塞尔基金会);Instituto de Telecomunicações(电信研究所);DIT Forlì - University of Bologna(福利大学迪提学院);Data Science Group - University of Hamburg(汉堡大学数据科学组);University of Trento(特伦托大学);LT 3 - Ghent University(根特大学LT 3)
Failure of contextual invariance in large language models
Network Science Institute, Northeastern University(网络科学研究所,东北大学);Center for Health Informatics Program, Boston Children’s Hospital(健康信息学计划中心,波士顿儿童医院);Dept. of Mathematics, City St George’s, University of London(伦敦大学城市圣乔治学院数学系);IT University of Copenhagen(哥本哈根IT大学)