Large language models are increasingly used in decision-making tasks that require them to process information from a variety of sources, including both human experts and other algorithmic agents. How do LLMs weigh the information provided by these different sources? We consider the well-studied phenomenon of algorithm aversion, in which human decision-makers exhibit bias against predictions from algorithms. Drawing upon experimental paradigms from behavioural economics, we evaluate how eightdifferent LLMs delegate decision-making tasks when the delegatee is framed as a human expert or an algorithmic agent. To be inclusive of different evaluation formats, we conduct our study with two task presentations: stated preferences, modeled through direct queries about trust towards either agent, and revealed preferences, modeled through providing in-context examples of the performance of both agents. When prompted to rate the trustworthiness of human experts and algorithms across diverse tasks, LLMs give higher ratings to the human expert, which correlates with prior results from human respondents. However, when shown the performance of a human expert and an algorithm and asked to place an incentivized bet between the two, LLMs disproportionately choose the algorithm, even when it performs demonstrably worse. These discrepant results suggest that LLMs may encode inconsistent biases towards humans and algorithms, which need to be carefully considered when they are deployed in high-stakes scenarios. Furthermore, we discuss the sensitivity of LLMs to task presentation formats that should be broadly scrutinized in evaluation robustness for AI safety.
Yet another algorithmic bias: A Discursive Analysis of Large Language Models Reinforcing Dominant Discourses on Gender and Race
Instituto de Estudos da Linguagem (IEL), Universidade Estadual de Campinas (UNICAMP)(语言研究学院(IEL),Campinas州立大学(UNICAMP));Instituto de Computação (IC), Universidade Estadual de Campinas (UNICAMP)(计算学院(IC),Campinas州立大学(UNICAMP));Instituto de Informática, Universidade Federal de Goiás (UFG)(信息学院,戈亚斯联邦大学(UFG))
Digital Skin, Digital Bias: Uncovering Tone-Based Biases in LLMs and Emoji Embeddings
University of North Texas(北德克萨斯大学);International Center for Artificial Intelligence Research and Ethics(国际人工智能研究与伦理中心);Southern Methodist University(南卫理公会大学);The University of Arizona(亚利桑那大学);The University of Alabama(阿拉巴马大学)
When Bias Backfires: The Modulatory Role of Counterfactual Explanations on the Adoption of Algorithmic Bias in XAI-Supported Human Decision-Making
Machine Learning Group, Bielefeld University(比勒菲尔德大学机器学习组);Research Center Trustworthy Data Science and Security of the University Alliance Ruhr, Faculty of Informatics, Technical University Dortmund(鲁尔大学联盟可信数据科学与安全研究所以及多特蒙德技术大学信息学院)
How optimal transport can tackle gender biases in multi-class neural-network classifiers for job recommendations?
Institut de Mathématiques de Toulouse(图卢兹数学研究所);CNRS(法国国家科学研究中心);Université de Toulouse(图卢兹大学);Institut de Recherche en Informatique de Toulouse(图卢兹计算机科学研究所);AKKODIS group(AKKODIS集团)
Laissez-Faire Harms: Algorithmic Biases in Generative Language Models
Young Data Scientists League(年轻数据科学家联盟);Stanford University(斯坦福大学);Georgia Institute of Technology(佐治亚理工学院);Schar School of Policy and Government & Department of Computer Science(政策与政府学院及计算机科学系);George Mason University(乔治·马歇尔大学)
Are All Genders Equal in the Eyes of Algorithms? -- Analysing Search and Retrieval Algorithms for Algorithmic Gender Fairness
Faculty for Computer Science and Mathematics, Hochschule München University of Applied Sciences(计算机科学与数学系,慕尼黑应用科学大学);Department of Statistics, LMU Munich(统计学系,慕尼黑大学);Munich Center for Machine Learning (MCML), LMU Munich(慕尼黑机器学习中心(MCML),慕尼黑大学)
Human-centered explanation does not fit all: The interplay of sociotechnical, cognitive, and individual factors in the effect AI explanations in algorithmic decision-making
University of Pittsburgh(匹兹堡大学);Northeastern University(东北大学);Syracuse University(Syracuse大学)