In this paper, we investigate the phenomena of "selection biases" in Large Language Models (LLMs), focusing on problems where models are tasked with choosing the optimal option from an ordered sequence. We delve into biases related to option order and token usage, which significantly impact LLMs' decision-making processes. We also quantify the impact of these biases through an extensive empirical analysis across multiple models and tasks. Furthermore, we propose mitigation strategies to enhance model performance. Our key contributions are threefold: 1) Precisely quantifying the influence of option order and token on LLMs, 2) Developing strategies to mitigate the impact of token and order sensitivity to enhance robustness, and 3) Offering a detailed analysis of sensitivity across models and tasks, which informs the creation of more stable and reliable LLM applications for selection problems.
Unveiling and Mitigating Bias in Mental Health Analysis with Large Language Models
Stanford University（斯坦福大学）；Meta Platforms, Inc.（元平台公司）；ETH Zürich（苏黎世联邦理工学院）；University Medical Center Utrecht（乌得勒支大学医学中心）；Leiden University Medical Center（莱顿大学医学中心）
DECASTE: Unveiling Caste Stereotypes in Large Language Models through Multi-Dimensional Bias Analysis
IBM Research, San Jose, CA, USA（IBM研究院，美国加利福尼亚州圣何塞）；Dartmouth College, Hanover, NH, USA（达特茅斯学院，美国新罕布什尔州汉诺威）；IBM Research, Hursley, Winchester, UK（IBM研究院，英国赫里福德郡温切斯特）；IBM Thomas J. Watson Research Center, Yorktown Heights, NY, USA（IBM托马斯·J·沃森研究中心，美国纽约州亚当斯）；IBM Research, Sao Paulo, Brazil（IBM研究院，巴西圣保罗）
RAcQUEt: Unveiling the Dangers of Overlooked Referential Ambiguity in Visual LLMs
Amsterdam UMC, Department of Medical Informatics（阿姆斯特丹大学医学中心，医学信息学系）；Center for Information and Language Processing, LMU Munich（信息与语言处理中心，慕尼黑大学）；Munich Center for Machine Learning (MCML), Munich（慕尼黑机器学习中心（MCML））；Institute for Logic, Language and Computation (ILLC), University of Amsterdam（逻辑、语言与计算研究所（ILLC），阿姆斯特丹大学）
Better Aligned with Survey Respondents or Training Data? Unveiling Political Leanings of LLMs on U.S. Supreme Court Cases
Technical University of Munich（慕尼黑技术大学）；Allen Institute for AI（艾伦人工智能研究所）；University of Washington（华盛顿大学）；LMU Munich & Munich Center for Machine Learning (MCML)（慕尼黑大学及慕尼黑机器学习中心）
Unveiling Linguistic Regions in Large Language Models
School of Computer Science, Fudan University（复旦大学计算机科学技术学院）；Institute of Modern Languages and Linguistics, Fudan University（复旦大学现代语言与语言学研究所）；Shanghai Collaborative Innovation Center of Intelligent Visual Computing（上海智能视觉计算协同创新中心）
Negation Blindness in Large Language Models: Unveiling the NO Syndrome in Image Generation
Aligarh Muslim University（阿里格尔穆斯林大学）；VIT Bhopal University（VIT博帕尔大学）；Nanyang Technological University（南洋理工大学）；Technical University of Munich（慕尼黑工业大学）；Imperial College London（伦敦帝国学院）；Edinburgh Napier University（爱丁堡龙比亚大学）
Unveiling Transformer Perception by Exploring Input Manifolds
Department of Environmental Science and Policy（环境科学与政策系）；Department of Computer Science（计算机科学系）；Department of Mathematics（数学系）；Department of Control and Computer Engineering（控制与计算机工程系）
Unveiling the Magic of Code Reasoning through Hypothesis Decomposition and Amendment
University of Science and Technology of China（中国科学技术大学）；Institute of Artificial Intelligence, Hefei Comprehensive National Science Center（合肥综合性国家科学中心人工智能研究院）
Unveiling Biases while Embracing Sustainability: Assessing the Dual Challenges of Automatic Speech Recognition Systems
IDIAP（IDIAP研究所）；MBZUAI（穆罕默德·本·扎耶德人工智能大学）；Erisha Labs（Erisha实验室）；Université de Lorraine（洛林大学）；CNRS（法国国家科学研究中心）；LORIA（洛里亚实验室）；INESC-ID（INESC-ID研究所）；IST（里斯本高等技术学院）；Universidade de Lisboa（里斯本大学）
AI in the Gray: Exploring Moderation Policies in Dialogic Large Language Models vs. Human Answers in Controversial Topics
IMDEA Networks Institute（IMDEA网络研究所）；Universidad Carlos III de Madrid（马德里卡洛斯三世大学）；University of Surrey（萨里大学）；King’s College London（伦敦国王学院）；VRAIN, Universitat Politecnica de Valencia（瓦伦西亚理工大学VRAIN研究所）
Exploring Embeddings for Measuring Text Relatedness: Unveiling Sentiments and Relationships in Online Comments
Bellarmine College Preparatory（贝拉明预备学院）；Barrington High School（巴林顿高中）；Los Osos High School（洛斯奥索斯高中）；California High School（加利福尼亚高中）；William Fremd High School（威廉·弗伦德高中）；Basis Independent Silicon Valley（贝西斯独立硅谷学校）；University of Maryland（马里兰大学）
